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Executive  Summary 


Recent  technological  advances  have  made  viable  the  implementation  of  intelligent  automation  in 
advanced  tactical  aircraft.  The  use  of  this  technology  has  given  rise  to  new  human  factors  issues  and 
concerns.  Errors  in  highly  automated  aircraft  have  been  linked  to  the  adverse  effects  of  automation  on  the 
pilot's  system  awareness,  monitoring  workload,  and  ability  to  revert  to  manual  control.  However,  adaptive 
automation,  or  automation  that  is  implemented  dynamically  in  response  to  changing  task  demands  on  the 
pilot,  has  been  proposed  to  be  superior  to  systems  with  fixed,  or  "static"  automation.  This  report  examines 
several  issues  concerning  the  theory  and  design  of  adaptive  automation  in  aviation  systems,  particularly  as 
applied  to  advanced  tactical  aircraft. 

An  analysis  of  the  relative  costs  and  benefits  of  "conventional"  (static)  aviation  automation  provides 
the  starting  point  for  the  development  of  a  theory  of  adaptive  automation.  This  analysis  includes  a  review  of 
the  empirical  studies  investigating  effects  of  automation  on  pilot  performance.  The  main  concepts  of 
adaptive  automation  are  then  introduced,  and  four  major  methods  for  implementing  adaptive  automation  in 
the  advanced  cockpit  are  described:  (1)  critical-event  logic;  (2)  pilot  workload  measurement;  (3)  pilot 
psychophysiological  assessment;  and  (4)  pilot  performance  modelling.  Important  theoretical,  technical ,  ana 
practical  issues  concerning  each  of  these  methods  are  discussed. 

To  be  effective,  adaptive  systems  should  be  designed  in  accordance  with  a  human-centered 
philosophy  of  automation.  Some  preliminary  guiding  principles  for  implementing  adaptive  automation  in  the 
cockpit  are  discussed.  Design  guidelines  must  currently  be  based  on  conceptual  analyses  because  of  the 
lack  of  empirical  evidence  concerning  the  impact  of  automation  (adaptive  and  otherwise)  on  the  human 
operators  in  aviation  systems.  Research  strategies  by  which  further  evidence  may  be  gained  are  discussed. 
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Introduction  and  Overview 

Mission  (unctions  in  modern  tactical  aircraft,  and  the  nature  of  the  threat  pilots  of  such  aircraft  face, 
have  grown  more  complex  over  the  years.  As  a  result,  there  has  been  a  commensurate  need  for  automation 
of  cockpit  functions  to  cope  with  this  increased  complexity.  Several  recommendations  have  been  made  in 
recent  years  to  automate  aircraft  systems  in  the  belief  that  automation  technology  will  improve  performance 
and  expand  tactical  capabilities  (National  Research  Council,  1982;  Defense  News,  1989).  Recent 
technological  advances  have  made  the  incorporation  of  intelligent  automation  a  viable  alternative  in  tactical 
aircraft.  The  Advanced  Technology  Fighter  (ATF)  and  its  Navy  version,  the  NATF,  are  examples  of  aircraft 
where  automation  technology  is  being  evaluated  for  incorporation  in  the  cockpit.  Other  Navy  efforts  which 
propose  to  use  advanced  automation  technology  include  the  Advanced  Technology  Crewstation  and 
KOALAS  programs  at  the  Naval  Air  Development  Center.  The  latter  is  an  intelligent,  man-in-the-loop, 
architecture  that  is  presently  being  demonstrated  as  a  means  to  integrate  real-time  expert  systems  into  a 
decision-support  system  for  E-2/F-14  applications. 

The  introduction  of  automation  technology  in  aircraft,  as  well  as  in  other  systems  such  as  process 
control,  has  given  rise  to  some  new  human  factors  issues  and  concerns.  For  example,  the  ability  of  the  pilot 
to  intervene  effectively  when  an  automated  subsystem  fails  is  one  of  the  key  issues  in  automated  cockpits 
(Wiener,  1988).  Other  difficulties  that  operators  of  automated  systems  may  face  include  loss  of  system 
awareness  and  manual  skills  degradation  (Norman,  Billings,  Nadel,  Palmer,  Wener,  &  Woods,  1988).  These 
kinds  of  problems  may  be  characteristic  of  complex  systems  in  which  automation  is  implemented  in  a  fixed  or 
"static”  manner.  In  contrast,  systems  in  which  automated  aids  are  implemented  dynamically,  in  response  to 
changing  task  demands  on  the  operator,  may  be  less  vulnerable  to  such  problems.  It  has  been  proposed 
that  systems  with  adaptive  automation1  are  superior  to  conventional  automation  because  they  provide  for 
regulation  of  operator  workload  and  vigilance,  maintenance  of  skill  levels,  and  task  involvement  (Hancock  et 
al.,  1985;  NADC,  1989;  Noah  &  Halpin,  1986;  Parasuraman  &  Bowers,  1987;  Rouse,  1976,  l988;Wickens 
&  Kramer,  1985).  More  generally,  systems  in  which  tasks  and  functions  are  allocated  flexibly  to  human 
operators  and  automated  subsystems  may  provide  for  more  effective  user-system  interaction. 

This  report  examines  several  issues  concerning  the  theory  and  design  of  adaptive  automation  in 
aviation  systems.2  The  emphasis  throughout  is  on  the  cognitive,  pilot-automation  interaction  issues  that 


1  Also  referred  to  as  "adaptive  aiding"  and  "adaptive  function  allocation." 

2  The  behavioral  literature  on  automation  issues  in  aviation  is  more  highly  developed  for  commercial  than  for 
military  aviation.  We  recognize  that  there  are  important  differences  between  the  two  spheres  of  aviation,  and 
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play  a  role  in  adaptive  systems3.  After  a  brief  introduction  to  the  ooncepts  and  history  of  aircraft  automation, 
we  consider  some  of  the  benefits  and  problems  associated  with  ’conventional*  aviation  automation.  The  few 
laboratory  studies  on  the  effects  of  automation  on  human  performance  are  then  reviewed.  Next,  adaptive 
automation  concepts  are  introduced,  and  the  four  main  methods  for  implementing  adaptive  automation  in 
the  advanced  cockpit  described.  Training  issues  for  effective  use  of  adaptive  systems  are  discussed  next, 
followed  by  a  discussion  of  preliminary  design  issues  and  guiding  principles  for  implementing  adaptive 
automation  in  the  cockpit.  Design  guidelines  must  currently  be  based  for  the  most  part  on  "armchair"  analysis 
and  on  opinions  based  on  previous  experience  with  automated  systems.  There  is  a  paucity  of  empirical 
evidence  in  these  areas,  particularly  concerning  the  efficacy  of  adaptive  automation.  As  we  identify  some 
of  these  conclusions,  we  emphasize  their  tentative  nature  and  suggest  research  strategies  by  which  further 
evidence  may  be  gained.  For  example,  there  is  little  empirical  work  showing  whether  pilots  can  use  adaptive 
aids  efficiently  in  the  cockpit4  and  whether  aircraft  with  adaptive  aids  (such  as  in  Lockheed’s  Pilot's  Associate 
program)  will  in  fact  be  immune  from  some  of  the  problems  that  have  arisen  with  more  conventional 
automation. 

What  Is  Automation? 

The  word  "automation”  has  so  been  widely  used  as  to  have  taken  on  a  variety  of  meanings. 
Nevertheless,  several  authors  have  discussed  the  concept  of  automation  and  tried  to  define  its  essence 
(Edwards,  1977;  Norman  and  Oriady,  1989;  Wiener,  1985 ).  The  American  Heritage  Dictionary  (1976) 
provides  a  simple  definition:  "automatic  operation  or  control  of  a  process,  equipment,  or  a  system."  This 
definition  is  general  but  straightforward.  Wiener  and  Curry  (1980)  provided  a  more  colorful  definition  of 
automation  as  "a  collection  of  tyrannical  self-serving  machines.  "  This  definition  emphasizes  that  people 
perceive  automation  from  different  perspectives.  Those  who  view  automation  from  a  mechanical  point  of 
view  consider  it  to  be  an  ensemble  of  "autonomous  machine  systems;"  whereas  those  who  are  concerned 


that  automation  issues  relevant  to  one  area  may  not  be  significant  for  the  other.  For  example,  the  problem  of 
low  levels  of  workload  and  boredom  that  have  been  reported  for  automated  commercial  airliners  (Wiener, 
1987)  may  be  less  of  a  problem  in  military  aircraft  (except,  for  example,  long-haul  transport  aircraft  such  as  the 
C-5  and  C-141). 

3  Interface  design  is  an  important  aspect  of  adaptive  system  design  that  will  not  covered  in  detail  in  this  report. 
Analyses  of  aviation  accidents  in  automated  aircraft  have  shown  that  a  number  of  errors  have  resulted  from 
poor  interface  design  (Norman  et  at..  1988).  The  representation  of  tasks  and  functions  that  are  carried  out  by 
automation  is  a  critical  element  of  adaptive  system  design. 

4  See,  however,  Geddes  (1986),  for  a  feasibility  study  of  the  use  of  adaptive  automation  in  the  F/A-18  fighter 


aircraft. 
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with  information  management  view  automation  as  a  new  information  system  with  enhanced  capabilities  (such 
as  those  found  in  diagnostic  systems)  (Woods,  1988). 

Definitions  of  automation  as  applied  to  aviation  systems  are  similarly  diverse.  At  one  extreme,  there  is 
a  tendency  to  consider  any  technology  addition  to  the  cockpit  as  automation.  At  the  other  extreme,  only 
devices  incorporating  expert  systems  have  been  considered  to  be  automation.  A  simple  rule  of  thumb  to 
guide  a  middle  ground  between  these  two  extremes  would  be  to  define  automation  as  :  a  device  that 
accomplishes  (partially  or  fully)  a  function  that  was  previously  carried  out  (partially  or  fully)  by  the  pilot.  It  is  also 
useful  to  think  of  automation  as  a  continuum  rather  than  as  atl-or-none.  The  concept  of  "levels  of  automation" 
has  been  discussed  by  a  number  of  authors  (McDaniel,  1988;  National  Research  Council,  1982).  At  one 
extreme  of  "total  manual  control"  the  function  is  continuously  controlled  by  the  pilot.  At  the  other  extreme  of 
"total  automation"  all  aspects  of  the  function  (including  its  monitoring)  are  delegated  to  a  machine  so  that  its 
operation  is  not  visible  to  the  pilot.  In  between  these  two  extremes  lie  different  degrees  of  participation  in  the 
function  by  pilot  and  automation.  McDaniel  (1988)  identified  10  such  levels.  For  example,  in  his  level  9, 
monitored  automation ,  the  automation  carries  out  a  series  of  operations  autonomously,  while  the  pilot  is  able 
to  monitor  the  operations  but  cannot  change  them.  However,  in  McDaniel's  level  6,  consent  automation,  the 
automation  displays  a  pending  action  to  the  pilot  and  requires  the  pilot's  consent  before  it  can  carry  out  the 
function.  Lower  levels  of  automation  would  require  the  user  to  designate  functions  and/or  define  functions. 

Adaptive  automation  has  been  proposed  as  a  means  for  further  increasing  the  number  and  flexibility 
of  levels  of  automation  in  the  cockpit.  One  outcome  of  this  concept  is  that  the  adaptive  system  could  select 
its  own  level  of  automation,  depending  upon  the  operating  environment  and  pilot  performance.  The 
feasibility  of  this  approach  and  its  impact  on  operator  and  system  performance  is  poorfy  understood.  At 
present,  the  only  consensus  in  the  design  of  adaptive  automation  systems  seems  to  be  that  the  philosophy  of 
the  operator's  role  in  the  system  will  be  critical.  Typically,  it  is  argued  that  the  operator  must  interact  with  the 
automation  as  an  executive,  and  as  such  provides  consent  to  the  level  of  automation  assumed  by  the  system 
(e.g.  McDaniel's  (1988)  level  6  automation  above).  This  has  been  deemed  crucial  in  the  Pilot's  Associate 
program  because  of  issues  related  to  maintenance  of  pilot  situational  awareness  and  pilot  acceptance. 

(System  actions  taken  without  pilot  concent  are  envisaged  only  when  failure  to  take  action  will  have  immediate, 
dire  consequences.)  Of  course,  this  itself  has  additional  implications  for  system  performance  because  of  the 
resultant  impact  on  pilot  workload.  These  implications  are  also  discussed  further  below. 

Automation  thus  encompasses  a  wide  range  of  technology,  from  simple  control  devices  to  complex, 
subsystems  involving  machine  intelligence.  As  these  technologies  have  been  introduced,  perceptions 
and  definitions  of  the  the  role  of  automation  have  changed  continuously.  For  example,  one  trend  that 
Rouse  and  Morris  (1986)  pointed  out  was  a  change  in  automation's  role  from  an  assisting  device  for  people  to 
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a  replacing  one.  The  rate  of  technological  progress  has  varied  from  domain  to  domain,  so  that  perceptions  of 
automation  have  varied  accordingly.  In  aviation,  automation  advances  have  been  particularly  rapid. 


Automation  In  Aviation:  Historical  Background 

Early  aircraft  automation  was  primarily  concerned  with  the  control  of  the  aerodynamic  surfaces  of  the 
aircraft.  Control  was  achieved  through  the  use  of  gyroscopic  devices  which  served  to  stabilize  the  aircraft 
attitude  by  activating  wing-warping  and  controlling  roll  and  pitch.  Gyrosoopic  stabilizers  have  been  widely 
used  to  control  aircraft  since  their  first  test  on  steam-powered  flying  machines  (Pailett,  1983).  At  the 
beginning  of  World  War  II  gyroscopic  devices  were  incorporated  into  the  “autopilots"  in  many  long-range 
aircraft.  Later,  the  navigational  functions  in  aircraft  were  facilitated  by  the  introduction  of  very  high  frequency 
omnirange  (VOR).  By  the  late  50's  sensors  provided  the  pilot  precise  data  regarding  magnetic  heading, 
altitude,  and  position.  In  the  60's  electronic  devices  were  introduced  which  provided  for  such  automatic 
flight  capabilities  as  automatic  landing,  control  of  power  and  flight  path,  and  so  on,  first  seeing  service  in  such 
aircraft  as  the  DC-10,  L-1011  and  other  commercial  aircraft. 

By  the  end  of  the  1970’s  a  further  step  was  taken  toward  sophisticated  automation  by  integrating  the 
area  navigation  system  in  the  autopilot.  It  was  at  about  this  period  of  time  that  the  first  safety  concerns  about 
automation  in  the  cockpit  became  the  topic  of  the  hour  (US  House  of  Representatives,  19,77).  Conclusions 
were  drawn  from  aviation  accidents  and  incidents  that  proper  human  factors  principles  should  be  considered 
while  introducing  automation  (Wiener,  1977;  Wiener  &  Curry,  1980).  Automation  technology  continued  to 
be  developed  and  introduced  largely  independently  of  these  concerns.  One  major  development  was  the 
implementation  of  more  flexible  electronic  CRT  displays.  This  was  followed  by  the  addition  of  automated 
system  management  devices.  These  new  automated  systems  enabled  vertical  as  well  as  horizontal 
automated  navigation  and  guidance,  and  also  provided  pilots  with  better  flight  path  precision  and  maximum 
fuel  economy.  The  new  Airbus  300  series  of  aircraft,  along  with  the  Boeing  757-767  and  the  McDonnell 
Douglas  MD-l  1  represent  the  most  highly  automated  commercial  aircraft  yet,  and  represents  the  culmination 
of  a  long  period  of  automation  technology  development. 

Automation  of  flight  control  functions  has  progressed  to  the  point  that  control  can  be  achieved 
when  manual  control  is  no  longer  possible.  In  all  the  aircraft  mentioned  above,  the  pilot  can  revert  to 
controlling  the  aircraft  manually  should  the  automation  fail.  In  certain  classes  of  automated  aircraft,  however, 
this  is  no  longer  possible,  because  the  aircraft  is  inherently  unstable  and  therefore  some  level  of  automation  is 
required  at  all  times.  One  example  are  the  experimental  Control  Configured  Vehicles  (CCV’s),  which  require 
control  inputs  on  a  second-by-second  basis  to  respond  to  rapidly  fluctuating  aerodynamic  forces:  manual 


Contract  No.  N62269-90-0022-5931 
Report  No.  NAWCADWAR-92033-60 


8 


control  ot  these  vehicles  is  impossible.  NASA's  Space  Shuttle  is  another  example.  In  such  aircraft,  multiple 
redundancy  is  built  in  to  the  automation  to  minimize  the  risk  of  system  failure  (e.g.,  the  Space  Shuttle  has  four 
(redundant)  flight  control  computers  on-line  simultaneously  as  well  as  an  off-line  backup). 

Automation  in  military  aircraft  has  followed  a  roughly  similar  path  to  that  in  civilian  automation,  with 
some  differences  A  comparison  of  older  and  modern  military  aircraft  will  reveal  many  superficial  similarities  to 
civilian  automation  trends.  For  example,  an  older-generation  fighter  such  as  the  F-4  had  numerous  lights, 
knobs,  dials,  and  other  simple  display  and  control  devices  in  place  of  the  multifunction  CRTs  of  modem  tactical 
aircraft.  However,  there  are  important  differences  between  the  two  spheres  of  aircraft  automation  in  a  few 
areas  related  to  the  special  functional  requirements  of  military  aircraft  (e.g.,  targeting  and  weapon 
deployment).  The  time  critical  nature  of  flight  functions  in  fighter  aircraft  has  also  imposed  special 
requirements  unique  to  military  automation.  Finally,  automation  trends  have  also  been  more  rapid  in  the 
civilian  than  in  the  military  sector  because  of  the  time  delays  inherent  in  funding,  procurement,  testing,  etc. 

In  general,  aviation  automation  has  progressed  from  (largely)  mechanically-based  systems  to 
computer-based  systems  incorporating  machine  ‘‘intelligence".  As  a  result,  the  functionality  made  available 
by  automation  has  advanced  from  very  limited  systems  to  highly  flexible  systems  in  which  the  functionality  of 
the  automation  may  be  chosen  according  to  the  user's  needs.  A  case  in  point  is  the  increasing  number  of 
modes  and  options  available  in  the  modern,  state-of-the-art  autopilot.  However,  the  case  studies  into 
accidents  involving  modern  aircraft  have  suggested  that  such  diversity  of  function  also  imposes  costs  in  terms 
of  operations  complexity  from  the  pilot’s  standpoint.  As  a  result,  designers  have  had  to  ask  if  in  fact  systems 
with  total  flexibility  are  in  fact  desirable? 

Aircraft  Automation:  Benefits  and  Problems 

Aircraft  automation  has  provided  important  benefits  in  a  number  of  areas,  including  fuel  economy, 
flight  control,  and  navigation.  The  extent  of  automation  technology  development  and  the  significance  of 
these  benefits  have  been  such  that  the  era  of  automated  aircraft  is  well  established— there  is  no  turning 
back.  But  along  with  these  benefits,  there  have  been  some  problems.  The  problems  have  been  not  merely 
those  normally  associated  with  the  introduction  of  any  new  technology,  but  fundamentally  new  problems.  It 
has  been  increasingly  recognized,  for  example,  that  while  aircraft  automation  has  reduced  or  eliminated  many 
kinds  of  error,  it  has  also  introduced  new  forms  of  error  that  did  not  exist  previously  (Wiener  &  Curry,  1980). 
Many  pilots  have  negative  reactions  to  automation  because  they  feel  it  has  increased  rather  than  reduced 
mental  workload  (Wiener,  1985, 1987).  Automation  sometimes  has  had  negative  effects  because  the 
resulting  human-machine  interface  is  altered  in  unpredictable  ways  (Adler,  1986;  Hirschhorn,  1984;  Noble, 
1984 ).  On  a  more  mundane  yet  nevertheless  important  note,  automation  of  some  operations  may  not 
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produce  expected  improvements  because  of  excessive  downtime  for  repair  and  calibration  (Blumenthal  and 
Dray,  1985).  In  an  extensive  evaluation  of  trends  in  commercial  aviation,  Norman  et  at.  (1988)  reached 
three  major  conclusions  concerning  the  development  of  aviation  automation  :  (1)  The  use  of  automation  has 
been  incremental  in  nature.  (2)  The  use  of  technology  has  resulted  in  the  partial  alienation  of  the  pilot.  (3) 
New  errors  have  been  introduced  with  the  use  of  new  technology.  The  impact  of  automation  on  the  pilots  is 
poorly  understood  in  part  because  of  how  it  has  been  implemented  in  present  day  systems. 

Developments  in  CRT  technology,  expert  systems,  and  paratief-distrib*  rted  processing  have  made  it 
possible  to  automate  many  high-level  functions  in  aircraft.  As  a  result,  the  major  issue  that  has  arisen  is  not  so 
much  whether  one  should  automate,  but  what  functions  should  be  automated  and  when  (Wiener  &  Curry, 
1980)?  Better  implementation  of  new  automation  requires  an  understanding  of  how  automation  can  be 
optimized,  that  is,  how  to  get  maximum  benefits  from  using  automation  while  correcting  all  possible  errors  that 
may  emerge  as  a  consequence  of  its  implementation.  The  pilot's  view  of  "good"  automation  is  represented 
by  Hoagland  (1984),  who  suggested  that  in  such  a  system  the  piiot  can  still  have  some  role  to  play  during 
critical  phases  of  an  automated  flight.  To  achieve  this,  important  information  for  the  pilot  is  needed. 

Hoagland  attributed  the  occurrence  of  several  errors  in  automated  aircraft  to  bad  system  design  and  to  the 
tendency  of  manufacturers  to  accept  any  request  to  automate.  Manufacturers  have  apparently  followed  the 
strategy  of  automating  where  possible  (i.e.,  when  the  technology  allows  it).  The  apparent  conflict  between 
the  manufacturers'  interests  and  the  goals  of  good  human  factors  design  needs  to  be  bridged. 

Consideration  of  this  issue  requires  an  analysis  of  the  benefits  and  problems  associated  with  automation. 

Benefits., of  Aircraft  Automation 

Boehm-Davis,  Curry,  Wiener  and  Harrison  (1983)  distinguished  between  two  types  of  automation 
benefits.  One  kind  of  advantage  accrues  because  the  automation  carries  out  functions  that  cannot  be 
achieved  otherwise  either  because  humans  cannot  perform  them  or  because  of  problems  of  cost,  safety  or 
time  needed  to  achieve  such  functions  manually.  For  example,  automation  has  allowed  aircraft  to  lly  in  all 
weather  conditions  in  a  safer  manner.  In  modem  tactical  fighters,  automation  is  necessary  to  help  the  pilot 
cope,  particularly  under  conditions  of  time  pressure,  with  an  increasingly  complex  threat  environment.  A 
second  type  of  benefit  offered  by  automation  occurs  because  automatic  systems  provide  better  performance 
than  humans. 

These  kind  of  benefits  can  be  perceived  from  two  points  of  view.  From  a  human  standpoint,  these 
benefits  are  expressed  in  such  terms  as  decreased  pilot  workload,  easier  aircraft  handling,  increased  safety, 
increased  passenger  comfort,  better  schedule  dependability,  and  so  on.  For  example,  automation  has 
reduced  cockpit  crew  size  in  commercial  aircraft  by  one-third  without  increasing  workload  significantly  for  the 
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remaining  two  crew  members  (Norman  et  at.  1988).  Reduced  crew  size  (without  reduced  system 
performance)  is  also  a  significant  benefit  from  a  systems  viewpoint.  Given  crew  training  costs  for  advanced 
tactical  aircraft,  tnis  represents  a  significant  benefit.  Other  system  benefits  of  automation  include 
consistency,  reliability,  and  cost-effectiveness.  Aircraft  automation  has  decreased  fuel  consumption, 
reduced  flight  times,  and  has  made  possible  operations  in  virtually  all  weather  conditions.  These  are 
significant  advantages  given  the  rapid  rise  in  air  traffic  and  the  increase  in  fuel  costs  in  the  last  two  decades. 

In  military  aviation,  automation  has  conferred  the  above-mentioned  benefits  as  well  as  others,  such  as 
improved  flight  performance,  expanded  tactical  capabili'y,  and  reduction  of  peak  periods  of  pilot  workload. 
Pilots  of  modern  fighter  aircraft  fly  in  such  a  high-workload  environment  that  they  would  be  overwhelmed  at 
certain  critical  times  in  the  flight  without  the  assistance  of  automation .  This  last  feature  is  unique  to 
automation  in  military  aircraft.  It  also  raises  a  potential  problem  that  leads  to  the  issue  of  the  costs  associated 
with  automation  that  is  discussed  in  the  next  section.  Given  the  high  workload  levels  that  the  fighter  pilot  has 
to  contend  with,  even  relatively  minor  functions  may  have  to  be  automated  for  no  other  purpose  than  to  keep 
pilot  workload  at  manageable  levels  and  to  enable  the  pilot  to  concentrate  on  the  more  important,  time-critical 
functions  (McDaniel,  1983).  While  this  may  be  an  effective  automation  strategy  is  some  instances,  it  could 
also  lead  to  certain  difficulties  for  the  pilot. 

Problems  of  Aircraft  Automation 

Aircraft  automation  has  not  been  without  its  unique  problems,  above  and  beyond  simply  the 
"teething"  problems  associated  with  any  new  technology. 

Automation-induced  problems.  This  type  of  problem  arises  when  the  anticipated  benefits  that 
automation  is  expected  to  confer  (such  as  those  mentioned  above),  do  not  occur.  One  example  of  such 
violations  of  anticipated  automation  benefits  occurs  when  the  pilot  is  faced  with  greater  workload  levels  with 
an  automated  system  than  existed  prior  to  the  automation,  despite  the  fact  that  the  automation  was  intended 
to  reduce  wotkload.  McDaniel  (1988)  described  how  this  problem  can  occur  in  the  context  of  fighter  aircraft 
automation: 

"If  the  automation  of  a  critical  function  is  not  perfectly  reliable,  the  pilot  will  need  to 
monitor  it  in  order  to  intervene  quickly  should  a  malfunction  occur.  If  the  pitot 
continuously  monitors  the  automation,  he  can  intervene  in  about  one  second.  If  he  is 
attending  to  another  task  when  the  malfunction  occurs,  the  reaction  time  will  be  several 
seconds  because  he  must  also  refresh  his  awareness  of  the  situation  as  well  as  detect 
that  a  malfunction  has  occurred,  what  has  malfunctioned,  and  what  to  do  about  it.  In  many 
situations,  the  malfunctioning  aircraft  cannot  survive  even  those  few  seconds.  As  a 
result,  a  pitot  dares  not  perform  a  second  non-critical  task  rather  than  monitor  the 
automated  critical  task.  So,  while  this  type  of  automation  permits  a  useful  task  to  be 
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accomplished,  it  does  nothing  to  free  the  pilot's  attention  resources  for  other  tasks.*  (p. 

837). 

The  paradox  is  that  implementing  automation  in  an  attempt  to  reduce  workload  may  actually  result  in 
increased  workload,  because  of  the  cognitive  workload  associated  with  monitoring  the  automation.  This 
increased  workload  can  be  a  problem  which  may  lead  to  major  difficulties  in  certain  situations.  Boehm-Davis 
et  al  (1983)  also  identified  passive  monitoring  as  a  related,  potentially  important  category  of  an  automation- 
induced  problem.  In  highly  automated  systems,  the  pilot  may  be  relegated  to  the  role  of  a  passive  monitor,  a 
task  for  which  humans  seem  ill  suited  (Parasuraman,  1987;  Wiener,  1987).  As  a  passive  monitor,  the  human 
operator's  function  is  simply  to  observe  the  operation  of  the  automated  system,  with  no  expectation,  or 
capacity,  to  affect  its  operation.  Such  a  role  is  not  a  stimulating  one  and  may  lead  to  boredom  or 
complacency,  as  well  as  to  possible  inefficiency  in  the  event  of  the  need  for  manual  control  under 
emergency  conditions  (Parasuraman,  1987).  Automation  may  also  induce  problems  because  of  training 
inadequacies  with  respect  to  monitoring.  Training  needs  for  automated  systems  are  complicated  because 
users  need  to  be  trained  as  monitors  for  situations  when  the  system  is  fully  automated  and  as  controllers 
when  the  manual  mode  is  on.  The  skills  required  for  both  capacities  may  conflict  with  one  another,  making  it 
difficult  to  decide  where  to  place  priorities  during  training.  Training  issues  are  considered  further  in  a  later 
section  of  this  report. 

Another  example  of  an  automation-induced  problem  is  the  ground  proximity  warning  system 
(GPWS)  which  produced  many  false  alarms  when  first  implemented  in  the  cockpit.  When  automation  does 
not  function  properly,  it  creates  additional  problems  of  credibility.  Users’  trust  of  a  particular  system  is  based 
on  past  experience  with  such  systems  as  well  as  their  knowledge  of  its  accuracy.  If,  for  instance,  pilots  are 
unsure  or  skeptical  of  the  reliability  of  an  automated  system  they  may  not  use  the  system  effectively. 

Problems  resulting  from  technology-centered  design.  These  concern  those  problems  that  arise  as  a 
result  of  the  failure  of  system  designers  to  consider  the  role  of  the  operator  of  automated  systems.  One 
major  problem  that  technology-centered  automation  design  is  thought  to  have  led  to  is  the  peripheralization 
of  the  pilot.  Some  examples  related  to  peripheralization  of  the  pilot  can  be  summarized  as  follows  (Norman  et 
al,  1988):  1)  Loss  of  situation  awareness,  in  which  the  pilot  does  not  have  an  adequate,  up-to-date  model  of 
the  relationship  of  his  aircraft  to  the  tactical  airspace  around  him.  2)  Loss  of  system  awareness,  in  which  the 
pilot  is  usually  unaware  of  the  system  capacities  and  limitations  as  well  as  its  behaviors  in  specific  situations.  3) 
Poor  interface  design.  A  good  example  of  this  type  of  complication  is  the  reprogramming  of  the  flight 
management  system  to  adapt  to  any  change  during  the  approach  phase.  This  phase  represents  such  a 
high  workload  level  for  pilots  that  they  usually  turn  the  automatic  system  off  and  use  a  manual  approach.  4) 
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Loss  of  manual  skills.  Most  plots  of  highly  automated  aircraft  often  manually  fly  their  aircraft  because  of  a  fear 
of  losing  their  basic  flight  skills  (Curry,  1985;  Wiener,  1988). 

It  is  now  well  recognized,  at  least  in  human  factors  circles,  that  these  problems  are  not  unique  to 
aircraft  systems,  but  have  arisen  in  several  other  systems  in  which  automation  has  been  implemented  without 
sufficient  regard  for  the  needs  and  characteristics  of  the  user  of  the  automated  system  (Woods,  1988).  This 
problem  has  been  attributed  to  what  is  called  technology  -centered  automation,  in  which  the  system  is 
primarily  designed  as  a  function  of  the  capabilities  of  the  available  technology.  The  alternative  that  has  been 
proposed  to  technology-centered  design  is  human-centered  automation  (NASA,  1989),  an  approach  that 
attempts  to  take  into  account  user  requirements  in  the  design  process.  This  approach,  and  its  application  to 
the  design  of  adaptive  automation,  is  considered  later  in  this  paper. 

Effects  of  Automation  on  Human  Performance 

Only  a  few  empirical  studies  have  been  carried  out  to  evaluate  the  effects  of  automation  on  human 
performance.  The  strategy  that  most  studies  have  followed  is  to  compare  performance  of  tasks  under 
automated  conditions  to  that  under  manual  conditions.  This  raises  an  immediate  difficulty.  Automating  a 
function  in  a  complex  system  such  as  the  cockpit  changes  the  nature  of  the  tasks  the  pilot  faces,  so  that  a 
comparison  between  automated  and  manual  performance  involves  a  number  of  different  variables,  not  all  of 
which  can  be  easily  controlled.  One  approach  to  dealing  with  this  problem  is  to  examine  manual  performance 
of  only  a  single  important  piloting  task,  such  as  system  monitoring  (e.g.,  failure  detection)  as  a  function  of 
whether  other  tasks  are  executed  manually  or  automatically. 

Effects  on  System  Monitoring 

As  discussed  earlier,  automation  may  have  adverse  effects  because  it  places  the  pilot  in  the  role  of  a 
passive  monitor.  Wiener  (1988)  has  given  a  number  of  examples  of  accidents  in  which  monitoring  failures  in 
automated  cockpits  were  a  major  contributing  factor.  In  contrast  to  passive  monitoring,  crew  members 
actively  involved  in  monitoring  are  expected  to  have  more  knowledge  of  the  system  and  “more  practice  at  a 
knowledge-based  level  of  behavior  (Idaszak  and  Hulin,  1989),  and  thus  be  better  equipped  to  detect  a 
system  failure.  While  this  reasoning  is  eminently  plausible,  only  a  few  studies  have  attempted  to  provide 
empirical  evidence  in  support  of  this  view. 

Wickens  and  Kessel  (1981)  carried  out  a  study  in  which  subjects  had  to  either ,  1)  monitor  only ,  or  2) 
both  monitor  and  control,  a  tracking  task.  The  monitoring  task  involved  a  sudden  change  in  the  control 
system  dynamics;  this  represented  the  "failure"  that  the  subjects  were  required  to  detect.  In  the  "manual" 
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condition  subjects  actively  controlled  the  tracking  and  monitored  at  the  same  time.  In  the  'passive 
automation'  condition  subjects  only  monitored  the  system  dynamics.  Speed  of  failure  detection  was  found  to 
be  significantly  slower  in  the  automated  than  in  the  manual  condition.  This  result  was  attributed  to  the 
subjects'  having  a  better  mental  representation  of  the  system  dynamics  in  the  manual  mode  (through  visual 
and  kinesthetic  feedback)  than  in  the  automated  mode. 

Idaszak  (1989)  carried  out  a  related  study  using  a  simulated  process  control  task.  Subjects  were 
required  to  monitor  the  system  and  respond  to  limits,  alarms,  and  deviations  in  process  parameter  values.  In 
the  active  monitoring  condition  subjects  controlled  the  process  and  monitored  it  at  the  same  time.  In  the 
passive  condition,  subjects  only  monitored  the  process  while  watching  a  video  of  nature  scenes.  Idaszak 
(1989)  found  that  the  active  group  monitored  better  than  the  passive  group:  active  operators  were  faster  at 
detecting  both  out-of-limits  conditions  and  alarms  than  the  passive  operators.  Idaszak  and  Hutin  (1989) 
suggested  that  active  participation  increases  the  operator's  workload  and  perceptions  of  task  difficulty  .and 
therefore,  active  participation  acts  as  a  source  of  motivation,  and  benefits  monitoring  performance.  However, 
one  problem  with  this  study  is  that  the  passive  group  was  required  to  answer  questions  about  the  video  they 
watched:  hence  it  is  likely  that  they  were  forced  to  allocate  resources  away  from  the  monitoring  task,  and  this 
diversion  of  resources  may  have  been  greater  than  in  the  active  group  doing  both  the  monitoring  and  the 
controlling  task.  It  is  therefore  questionable  whether  the  passive  condition  used  in  this  study  is 
representative  of  the  passive  monitoring  that  automation  in  the  cockpit  induces. 

These  results  provide  some  empirical  support  for  the  contention  that  automation  impairs  manual 
monitoring  performance  under  emergency  conditions,  a  so-called  'automation  deficit"  (Wiener  &  Curry,  1980). 
However,  the  studies  are  by  no  means  conclusive  in  demonstrating  such  a  deficit.  Moreover,  more  needs  to 
be  learned  about  the  characteristics  of  this  deficit.  Is  it  a  general  phenomenon  affecting  all  aspects  of 
cognitive  performance?  What  is  the  minimal  level  of  manual  involvement  required  for  the  deficit  not  to  occur, 
and  what  cognitive  processes  are  involved?  The  evidence  available  to  date  suggests  that  the  phenomenon 
may  not  be  a  very  general  one  and  may  only  occur  in  continuous  tracking  tasks.  Fuld,  Uu,  and  Wickens 
(1987)  carried  out  an  experiment  that  was  designed  similarly  to  that  by  Wickens  and  Kessel  (1981),  but 
involved  a  continuous  visual  sorting  and  queuing  task  instead  of  a  tracking  task.  Subjects  had  to  assign  an 
incoming  "customer"  to  one  of  three  queues  for  service.  In  the  manual  mode,  subjects  had  to  make 
assignments  (by  pressing  one  of  three  buttons,  one  for  each  queue)  as  well  as  monitor  their  own 
assignments  and  press  another  key  if  they  detected  an  error.  In  the  automated  mode  subjects  were  told  that 
the  queuing  assignments  would  be  made  by  the  computer  and  that  their  task  was  only  to  detect  wrong 
assignments.  In  fact,  the  ‘automation’  condition  was  simulated  by  giving  each  subject  a  replay  of  their  earlier 
manual  performance:  this  was  done  to  control  tor  visual  display  differences  between  the  automated  and 
manual  modes.  Fuld  et  al.  found  that,  in  contrast  to  Wickens  and  Kessel  (1981 ),  monitoring  performance 
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(sensitivity  (A')  in  detecting  wrong  assignments)  was  better  in  the  automated  condition  than  in  the  manual 
condition.  However  this  effect  may  have  been  influenced  by  the  control  method  that  Fuld  et  al.  used. 
Because  subjects  performed  the  automated  condition  after  the  manual  condition  and  received  the  same 
sequence  of  stimuli,  they  may  have  benefitted  from  seeing  difficult  errors  (that  they  made  initially  in  the 
manual  condition)  again  in  the  automated  condition,  whereas  subjects  seeing  such  errors  for  the  first  time  may 
not  have  detected  them. 

Effects  on  Situational  Awareness 

Kibbe  and  Wilson  (1989)  conducted  three  experiments  examining  the  effect  of  automation  on  the 
retention  of  information  from  cockpit  displays.  A  simulated  electronic  warfare  (EW)  display  was  used  in  each 
study,  incorporating  a  threat  detection  task  and  a  tracking  task.  Subjects  were  queried  at  the  end  of  a 
scenario  on  various  aspects  of  the  threat  detection  task,  as  an  indirect  measure  of  their  “situation  awareness." 
Kibbe  and  Wilson  hypothesized  that  automation  impairs  situational  awareness  because  the  operator  is  not 
actively  involved.  Manual  and  automated  modes  of  performance  were  compared  to  test  this  hypothesis. 
While  a  number  of  expected  results  related  to  single  versus  dual-task  performance  were  obtained,  there  were 
no  significant  effects  of  system  mode  (automated  or  manual)  on  the  amount  of  information  retained  from  the 
display.  Thus  the  experiment  failed  to  find  any  effect  of  automation,  positive  or  negative.  A  minor  finding 
was  that  some  subjects  intervened  more  than  others  in  the  automated  system  and  switched  to  the  manual 
mode  when  the  automation  failed.  This  behavior  (intervention)  was  not  linked  to  any  changes  in  the 
performance  of  either  one  of  the  two  tasks. 


Relating  Costs  and  Benefits  .of  Aulamaligj] 

One  of  the  few  empirical  evaluations  of  the  relative  merits  and  costs  of  automation  technology  with 
respect  to  pilot  performance  was  carried  out  by  Bortolussi  and  Vidulich  (in  press).  A  helicopter  simulation  was 
designed  to  examine  the  interaction  between  voice-activated  controls  and  altitude-hold  automation.  Pilots 
conducted  simulated  missions  which  included  hover,  cruise  and  ground  attack  phases.  Voice-activated 
controls  were  used  instead  of  manual  controls  for  half  of  the  flights  ( for  weapon  selection  and  data-burst 
transmission).  Results  showed  that  voice-activated  controls  helped  improve  hover  performance  during  data- 
burst  hover.  This  improvement  was  interpreted  as  an  outcome  of  a  small  reduction  in  the  competition  for 
manual  resources.  On  the  other  hand,  and  as  indicated  by  the  pilot's  subjective  rating,  pilots  preferred 
manual  control  activation  for  high  priority  flight  tasks,  such  as  weapon  selection.  The  pilots  reported  that 
workload  increased  whi'e  using  voice-activated  controls,  primarily  due  to  their  unfamiliarity  with  the  systems 
and  their  reliability.  The  attitude-hold  automation  improved  performance  on  tasks  and  pilots  reported  low 
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workload.  However,  pilots  were  significantly  slower  in  responding  to  unexpected  events  during  hover.  This 
was  interpreted  as  a  significant  cost  associated  with  automation.  Note  that  this  'automation  deficit'  effect, 
like  that  of  Wickens  and  Kessel  (1981)  again  involved  a  tracking  task.  Further  research  is  needed  to 
determine  whether  this  effect  is  indeed  restricted  to  tracking  or  motor  tasks,  or  whether  it  is  a  more  general 
phenomenon. 


Adaptive  Automation  Concepts 

While  automation  has  offered  new  techniques  to  solve  various  problems  encountered  in  advanced 
aircraft,  it  has  brought  new  questions  about  the  role  and  and  effectiveness  of  the  pilot.  At  the  same  time,  as 
indicated  above,  the  benefits  associated  with  automation  have  been  accompanied  by  certain  costs.  It  has 
been  proposed  that  one  reason  that  aircraft  automation  has  led  to  certain  difficulties  is  because  existing 
approaches  to  automation  implementation  have  been  "static"  rather  than  "dynamic"  (NADC,  1989).  In  this 
view,  it  has  been  proposed  that  many  of  the  benefits  of  automation  can  be  maximized  and  the  costs 
minimized  if  automation  is  implemented  in  an  adaptive  manner  rather  than  in  an  all-or-none  fashion  (NADC, 
1989;  Rouse,  1988).  But  this  approach  to  automation  has  yet  to  be  demonstrated  reliably,  either  in  the 
laboratory,  simulator,  or  cockpit.  More  research  is  required  to  test  the  concept,  and.  most  importantly,  to 
acquire  empirical  evidence  for  its  effects  on  pilot  and  system  performance. 

Adaptive  processes  are  thought  to  allow  synergistic  communication  between  the  pilot  and  the 
support  automation.  For  example,  the  pilot  can  actively  control  a  process  during  moderate  workload  and 
allocate  this  function  to  an  automated  subsystem  during  peak  workload  if  necessary.  Fully,  or  statically 
automated  processes,  on  the  other  hand,  can  impact  negatively  on  pilot  workload  and  lead  to  a  loss  of  system 
awareness  (Norman  et  at..  1988).  Adaptive  automation  is  thus  believed  to  allow  the  advantages  of 
automation  to  be  realized  while  maintaining  pilot  involvement  in  the  system.  Currently  ongoing  research  at  a 
number  of  laboratories  will  determine  whether  this  assertion  is  true. 

The  adaptive  automation  ooncept  is  not  a  new  one,  having  been  proposed  several  years  ago 
(Rouse,  1976).  The  idea  of  automation  that  is  variable  rather  than  fixed  also  falls  naturally  out  of  the  "levels  of 
automation"  concept  discussed  earlier.  It  is  also  related  to  the  classic  problem  in  human  factors  of  allocation 
of  function  between  humans  and  machines  (Fitts,  1961;  Jordan,  1963;  Parasuraman,  1990;  Singleton, 

1971).  Usually  the  decision  as  to  which  operations  should  be  carried  out  by  computer  is  determined  by 
whether  automation  is  feasible  or  not.  Because  decisions  about  task  allocation  cannot  always  be  made  based 
on  stereotypical  characteristics  of  human  and  computer  capabilities,  the  need  for  an  adaptive  computer  aid 
that  responds  to  task  demands  and  operator  performance  becomes  a  logical  design  alternative  for  increasing 
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system  effectiveness  (Hancock  et  al.,  1985;  Morris,  Rouse  and  Ward,  1984).  While  previous  attempts  to 
solve  the  allocation  of  function  problem  met  with  mixed  results  (e.g.,  see  Jordan,  1963),  developments  in 
computer  technology,  particularly  in  machine  intelligence  and  parallel-distributed  processing,  have  led  to 
greater  optimism  for  the  success  of  adaptive  systems. 

In  addition,  the  emergence  of  expert  system  technology  has  facilitated  the  development  of  systems 
design  tools  that  allow  a  better  understanding  of  the  operator  (expert)  decision  process.  Given  a  greater 
understanding  of  the  strategies  and  knowledge  of  the  pilot,  it  should  be  possible  to  design  adaptive 
automation  that  can  work  with  as  opposed  to  lor  the  pilot. 

At  the  moment,  however,  adaptive  systems  provide  only  a  promise  for  an  improved  pilot-vehicle 
interlace;  whether  that  promise  is  realized  will  depend  first  upon  the  satisfactory  resolution  of  a  number  of 
issues  related  to  the  methods  by  which  adaptive  aids  are  implemented.  Furthermore,  it  is  important  that 
these  issues  inform  the  design  methodologies  that  are  used  in  actually  building  adaptive  automation  into  the 
cockpit.  In  the  final  section  of  this  paper  some  prospective  design  principles  are  discussed.  Finally,  other 
issues  such  as  user  acceptance  and  the  ease  of  use  of  adaptive  aids,  which  may  appear  ancillary  at  first,  may 
also  ultimately  determine  the  success  of  adaptive  systems.  As  Rouse  (1988)  put  it:  "When  we  first 
introduced  the  (adaptive  automation)  concept  to  aircraft  pilots  in  1974-75  their  comments  ranged  from 

thoughtful  cautions  to  outright  ridicule.  More  recently  (1987) . we  have  found  pitots  to  be  quite  positive 

but  still  conservative"  (p.  442). 

One  of  the  key  issues  in  adaptive  automation  concerns  the  method  by  which  adaptation  is 
implemented.  In  other  words,  how  is  the  adaptation  to  be  done?  What  properties  (of  the  pilot,  the  task 
environment,  or  both)  should  the  adaptive  logic  detect  and  respond  to?  What  tasks  should  in  fact  be 
automated,  and  when?  A  number  of  different  schemes  for  adaptive  task  allocation  have  been  proposed,  from 
methods  based  on  dynamic  measurement  of  operator  workload  or  psychophysiological  states  to  optimal 
performance  models  that  incorporate  rule  bases  on  operator  resources,  strategies  and  intentions  (Hancock  et 
al.,  1985;  Parasuraman  &  Bowers,  1987;  Reising,  1985;  Riley,  1987;  Rouse,  1977;  Rouse  &  Rouse,  1988). 
These  automation  control  strategies  are  considered  next. 

Adaptive  Automation  Control  Methods 


A  number  of  proposals  for  adaptive  systems  have  been  proposed  over  the  years.  Rouse  (1988)  has 
categorized  all  adaptive  schemes  as  based  on  either  measurement  or  modeling  of  the  operator.  However, 
a  third  category  for  invocation  of  adaptive  processes,  based  purely  on  mission  doctrine,  is  also  possible.  This 
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method  is  called  critical-event  logic  and  can  be  implemented  without  the  need  for  measuring  or  modelling 
the  pilot's  performance  (Barnes  &  Grossman,  1985).  All  three  methods  are  considered  in  more  detail  below. 

The  advantage  of  critical-event  logic  is  that  it  can  be  tied  closely  to  actual  military  tactics  and  doctrine. 
Its  disadvantage  is  its  insensitivity  to  actual  pilot  performance  or  workload.  The  measurement  technique 
attempts  to  overcome  this  limitation.  In  this  method,  various  pilot  mental  states  (workload,  vigilance, 
strategies,  intentions,  etc)  may  be  measured  and  these  measures  then  fed  to  some  adaptive  logic. 
Alternatively  pilot  states  and  performance  may  be  modelled  theoretically,  with  the  adaptive  algorithm  being 
driven  by  the  model  parameters.®  The  measurement  and  modelling  methodologies  each  have  merits  and 
disadvantages.  Measurement  has  the  advantage  of  being  an  "on-line"  technique  that  can  potentially 
respond  to  unpredictable  changes  in  pilot  cognitive  states.  However,  this  method  is  only  as  good  as  the 
sensitivity  and  diagnosticity  of  the  measurement  technology.  Modelling  techniques  have  the  advantage  that 
they  can  be  implemented  off-line  and  easily  incorporated  into  rule-based  expert  systems.  However,  this 
method  is  only  as  good  as  the  theory  behind  the  model,  and  many  models  may  be  required  to  deal  with  all 
aspects  of  pilot  performance  in  a  complex  task  environment.  Hybrid  systems  that  combine  measurement  and 
modelling,  or  critical-event  logic  and  measurement,  or  other  possible  combinations  of  these  methods,  may 
optimize  their  relative  benefits. 

The  adaptive  scheme  that  is  simplest  conceptually,  and  relatively  easy  to  implement  is  one  employing 
critical-event  logic.  An  adaptive  scheme  that  is  also  simple  conceptually,  but  not  necessarily  the  easiest  to 
implement,  is  one  based  on  assessment  of  operator  workload.  An  operator  workload  based  system  has 
appeal  because  the  primary  raison-d'etre  for  adaptive  automation  is  the  regulation  of  mental  workload. 
Assessment  of  the  pilot  may  also  extend  to  psychophysiological  states.  These  are  both  measurement 
methods.  Finally,  optimal  performance  models  of  the  pilot  may  also  be  used  as  the  basis  for  adaptation.  We 
discuss  each  of  these  methods  in  turn. 

CriticahEyflQlJLooic 

In  this  method  the  implementation  of  automation  is  tied  to  the  occurrence  of  specific  tactical  events 
(Barnes  &  Grossman,  1985).  For  example,  the  beginning  of  a  "pop-up*  weapon  delivery  sequence  may  lead 
to  the  automation  of  all  aircraft  defensive  measures.  This  method  of  automation  is  adaptive  because  if  the 
critical  events  do  not  occur,  the  automation  is  not  invoked.  Such  an  adaptive  automation  method  is  inherently 
flexible  because  it  can  be  tied  to  current  military  doctrine  during  mission  planning. 


5  For  example,  model-based  adaptation  is  used  in  the  Pilot's  Associate. 
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Barnes  &  Grossman  (1985)  identified  three  types  of  adaptive  logic  within  this  general  scheme:  (1) 
Emergency  togic,  in  which  a  control  process  is  executed  without  pilot  intervention  initiation  or  intervention 
(tor  example,  SAM  detected  within  critical  envelope).  (2)  Executive  logic,  in  which  the  subprocesses  leading 
up  to  the  decision  to  activate  the  process  are  automatically  invoked,  with  the  final  decision  requiring  the  pilot's 
input.  (3)  Automated  display  logic,  in  which  all  non-critical  display  findings  are  automated  to  prepare  for  a 
particular  event,  so  that  the  pilot  can  concentrate  on  the  most  important  tasks.  Interestingly,  this  last  method 
is  also  implicitly  a  workload-related  method,  although  it  is  not  workload -adaptive,  because  it  assumes  that  the 
pilot's  workload  will  always  increase  beyond  his  upper  limit  when  the  critical  event  occurs. 

To  be  successful,  systems  based  on  critical-event  logic  would  require  an  intelligent  interface  to  allow 
the  pilot  to  set  up  tailored  rule  bases  before  the  mission.  Similar,  although  simpler,  adaptive  methods  art 
currently  implemented  for  AEGIS  and  the  HARM  missile  system.  The  AEGIS  system  is  based  on  doctrine 
which  the  ship  commander  sets  for  the  current  operational  environment.  Doctrine  is  a  small  rule  base  in  a 
production  system  that  determines  how  the  AEGIS  system  will  operate  in  a  combat  environment.  Three 
modes  of  operation  are  implemented:  (1)  Manual,  where  the  system  is  fully  controlled  by  the  operator.  (2) 
Automatic  special,  in  which  a  critical  event  (e.g.,  an  enemy  aircraft  at  a  specified  range)  would  automatically  set 
up  a  weapon-delivery  process  (e.g.,  prepare  standard  missile  to  fire):  however,  the  operator  would  press  the 
fire  button;.  (3)  Fully  automated,  where  the  need  for  a  short  reaction  time  requires  automatically  involving  ship 
defensive  systems  without  operator  intervention  (Armageddon). 

The  problem  with  these  systems  is  that  they  are  relatively  unsophisticated  and  are  unresponsive  to 
actual  operator  workload  or  performance.  It  is  even  possible  that  their  rule  bases  have  unforeseen,  interactive 
consequences  in  a  complex  environment.  In  both  the  automatic-special  and  fully-automated  systems,  the 
rule  base  is  kept  small  and  decidedly  "non-expert"  to  allow  the  planner  to  understand  the  consequences  of 
the  rules.  The  operator  workload  is  reduced  because  decisions  can  made  off-line  prior  to  combat.  At  the 
same  time,  the  user  (based  on  local  conditions)  and  an  Al  "expert"  tailors  the  rule  base  controlling  the  adaptive 
logic. 


In  conclusion,  adaptive  automation  based  on  critical-event  logic  probably  represents  the  "baseline" 
adaptive  system.  These  systems  are  simple  and  relatively  unsophisticated  by  design.  Important 
measurement  and  modelling  problems  must  be  overcome  before  such  adaptive  systems  can  be  controlled  by 
truly  intelligent  logic. 
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Dynamic  Assessment  of  Pilot  Mental  Workload 

Rouse  (1977)  first  described  a  conceptual  scheme  for  dynamic  allocation  of  tasks  in  a  multi-task 
situation  based  on  the  operator's  moment-to-moment  level  of  workload.  Mental  workload-based  methods  of 
adaptation  lie  at  the  heart  of  the  basic  rationale  for  implementing  automation  in  an  adaptive  rather  than  fixed 
manner.  The  goal  of  adaptive  automation  is  to  regulate  pilot  workload  levels  around  some  optimal  level  that  is 
neither  too  low  nor  too  high.  Critical  to  workload-based  adaptation  is  the  availability  of  a  reliable  assessment 
of  pilot  mental  workload  obtained  in  real-time  (second-to-second  or  minute-to-minute,  depending  upon  the 
specifics  of  the  mission).  The  development  of  accurate,  non-obtrusive  methods  for  assessing  workload  and 
incorporating  them  into  real-time  algorithms  is  essential  if  workload  is  to  serve  as  the  basis  for  determining 
what  functions  are  to  be  automated  and  when. 

Operators  of  complex  systems  such  as  fighter  aircraft  generally  work  in  a  very  high  workload 
environment.  However,  automation  of  a  function  may  be  useful  or  even  essential  to  the  pilot  at  one  moment 
but  not  at  another.  Workload  levels  fluctuate  from  moment  to  moment  and  at  different  mission  phases. 

Pilots  may  be  able  to  achieve  very  high  performance  levels  but  only  at  the  cost  of  high  mental  workload  and. 
perhaps,  the  neglect  of  less  critical  tasks.  As  a  result,  if  this  high  level  of  workload  has  to  be  sustained  for  long 
periods  of  time,  performance  degradation  may  result.  Alternatively,  performance  may  deteriorate  with  the 
addition  of  other  minor  tasks.  Furthermore,  different  pilots  will  use  differing  strategies  to  cope  with  the 
demands  of  multiple  tasks  under  time  pressure,  which  will  require  the  development  of  custom  tailored 
adaptive  automation  algorithms  if  the  system  is  to  be  compatible  with,  and  complement,  the  strengths  and 
weaknesses  of  individual  pilots.  Moreover,  individual  differences  in  pilot  capabilities  will  influence  the 
response  to  multiple  task  demand s-the  superior  pilot  will  have  sufficient  resources  left  to  cope  with  other 
tasks  whereas  the  average  plot  will  be  operating  at  peak  workload.  Hence  any  "static"  automation  system 
scheme  that  implements  automation  on  the  basis  of  the  required  workload  demand  on  an  average  pilot  will 
penalize  and  consequently  underutilize  the  skills  of  the  superior  pilot. 

For  each  of  these  reasons,  an  automation  scheme  that  is  responsive  to  individual,  momentary 
workload  levels,  will  serve  to  regulate  workload  and  optimize  performance  regardless  of  specific  tactical 
demands  and  pilot  capabilities.  Note  that  wo/kbad  is  adapted  to,  not  performance  (although  a  performance 
measure  may  be  used  to  assess  workload) .  Given  that  the  technical  problem  of  dynamic  workload 
measurement  can  be  solved,  adaptive  systems  should  achieve  the  goal  of  optimal  mental  workload  to  be 
pursued  in  dynamic,  real-time  environments. 
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Sensitivity  and  Diaonosticitv.  The  efficacy  of  workload-based  adaptation  is  dependent  upon  the 
sensitivity  and  diagnosticity  of  the  workload  measurement  technology6.  The  research  literature  on  workload 
measurement  techniques  has  grown  to  voluminous  proportions  in  recent  years  (Hancock  &  Meshkati,  1988; 
Moray,  1979;  Wickens,  1984).  A  number  of  sensitive  measures  have  been  identified,  with  the  most  reliable 
measures  being  based  on  variations  of  the  secondary-task  method.  Other  categories  of  workload 
measurement  technology  involve  primary-task  measures,  psychophysiological  indices,  and  subjective 
measures.  However,  it  has  proved  more  difficult  to  find  measures  that  are  both  sensitive  to  momentary 
fluctuations  in  workload  and  can  be  implemented  easily  in  the  cockpit. 

In  addition  to  sensitivity,  a  workload  metric  must  be  diagnostic  to  be  of  use  in  adaptive  automation 
(Wickens,  1984;  Wickens  &  Kramer,  1985).  In  other  words,  the  measure  must  indicate  what  component  of 
mental  workload  is  under  or  over-stressed.  For  example  a  pilot  heavily  loaded  with  visual  display  information 
may  nevertheless  be  able  to  cope  effectively  with  auditory  inputs.  A  workload  method  that  did  not  provide 
some  measure  of  diagnosticity  may  therefore  erroneously  trigger  the  adaptive  algorithm  to  off-load  the  pilot. 
Unfortunately,  while  considerable  progress  has  been  made  in  developing  sensitive  workload  measures,  there 
has  been  less  progress  in  developing  h'ghly  diagnostic  workload  measures. 

Dynamic  Aspects  of  Workload.  Hancock  and  Chignell  (1988)  have  recently  proposed  a  workload 
model  in  the  context  of  adaptive  systems.  In  contrast  to  most  existing  approaches  to  workload,  which  tend  to 
be  "static"  and  driven  solely  by  task  structure  (e.g.  display  and  response  modes),  their  model  is  a  dynamic  one 
which  recognizes  that  operators  pursue  active  strategies  to  regulate  their  workload  in  response  to  task 
demands.  Such  regulatory  strategies  include,  pre-planning,  task  scheduling,  and  partial  task  postponement. 
The  time  available  for  carrying  out  functions  in  a  multitask  situation  plays  an  important  role  in  their  model. 
According  to  Hancock  and  Chignell,  "successful  performance  depends  upon  the  reconciliation  of  the 
complexity  and  difficulty  of  the  imposed  task  with  the  time  within  which  the  goal  must  be  achieved"  (p  649). 
Their  hypothesis  is  that  task-induced  mental  workload  is  a  joint  function  of  the  current  subjective  distance 
from  a  desired  goal,  the  real  time  needed  to  reach  that  goal,  and  the  level  of  operator  effort  required  to 
achieve  the  goal,  given  the  time  available.  These  three  factors  can  be  represented  in  a  three-dimensional 
surface  by  which  momentary  workload  c  be  assessed.  Hancock  etal.  (1989)  carried  out  an  experimental 
test  of  the  model  using  a  target  acquisition  task  (i.e.  a  Fitts'  Law  task)  with  targets  that  varied  continuously  in 

6  A  number  of  other  workload-measurement  criteria  are  also  relevant  to  adaptive  automation,  including  the 
intrusiveness  of  the  method,  its  acceptability  to  the  pilot,  etc.  See  Eggermeir  and  O'Donnell  (1986)  for  a 
discussion. 
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size.  Performance  measures  and  subjective  workload  indices  gave  generally  good  fits  to  the  predictions  of 
the  model. 

The  adaptive  aspect  of  Hancock  and  Chignell's  (1988)  model  is  similar  to  other  adaptive  automation 
schemes.  Adaptation  (task  loading,  task  automation,  task  aiding,  etc)  requires  an  evaluation  of  both 
demands  on  workload  and  the  available  operator  resources.  In  Hancock  and  Chignell's  scheme,  this  could 
be  achieved  using  either  measurement  or  modelling  ( which  they  term  as  prediction).  Once  information 
about  the  level  of  mental  workload  is  provided,  the  adaptivity  of  the  system  may  be  tailored  to  the  needs  of  a 
human  operator.  They  suggest  that  the  first  step  is  to  develop  a  criterion  for  adaptivity.  This  is  accomplished 
in  two  stages:  (1)  Estimating  how  far  workload  is  (lower  or  higher)  from  the  optimum;(2)  Generating  a 
diagnosis  of  information  so  as  to  evaluate  the  load  on  particular  cognitive  structures. 

The  proposed  model  works  as  follows:  first  the  task  is  defined,  structured  and  subtasks  allocated  to 
either  an  automated  subsystem  or  to  the  operator.  Next,  the  operator's  effort  is  compared  with  the  task 
difficulty  so  as  to  assign  a  criterion  for  adaptivity.  The  criterion  can  be  expressed  as  a  measure  of  mental 
workload,  a  measure  of  primary  task  performance  or  a  combination  of  both.  Once  the  criterion  is  defined,  an 
adaptive  policy  is  implemented.  In  other  words,  the  criterion  remains  dynamic  and  changes  every  time  there 
is  an  alteration  in  the  operator's  performance  and/or  task  complexity.  The  adaptive  system  is  defined  as 
alteration  of  task  components  in  order  to  improve  future  measures  of  the  criterion.  In  other  words,  the 
adaptive  system's  role  is  to  minimize  the  difference  between  present  requirement  and  accessible  capacity. 
Adaptivity,  according  to  this  workload-based  measurement  method,  can  be  achieved  through  three  main 
procedures:  by  adjusting  the  allocation  of  subtasks  between  the  operator  and  the  automation;  by  adjusting 
the  structure  of  the  task;  and  by  redefining  the  task. 

Dynamic  PsvchoDhvsioiooical  Assessment 

Psychophysiologica!  states  of  the  pilot  may  provide  additional  information  that  can  be  tapped  for 
control  of  adaptive  systems.  Technology  is  available  to  measure  a  number  of  physiological  signals  from  the 
pilot,  from  autonomic  measures  such  as  pupillary  dilation  (Beatty,  1986)  and  heart  rate  (Jenkins,  1986)  to 
central  nervous  system  measures  such  as  the  EEG  and  event-related  potentials  or  ERPs  (Donchin,  Kramer, 

&  Wickens,  1986;  Parasuraman,  1990),  as  well  as  measures  such  as  eye  scanning  and  fixations .  There  is 
now  a  substantial  literature  indicating  that  different  psychophysiological  measures  can  be  obtained  of  such 
covert  mental  activities  as  attention  allocation,  memory  look  up,  and  response  preparation.  However,  given 
the  greater  cost  and  unproven  reliability  of  psychophysiological  measures  in  the  cockpit,  it  is  pertinent  to  ask 
why  they  should  be  used.  What  advantages  do  psychophysiological  measures  offer  over  behavioral 
methods,  if  any? 
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Psychophysiological  measures  offer  two  main  advantages  over  other  measures.  In  certain 
applications,  these  advantages  may  be  sufficiently  high  to  overcome  the  disadvantages  of  cost,  user 
acceptance,  etc.  associated  with  these  measures.  First,  psychophysiological  measures,  unlike  most 
behavioral  measures  (with  the  exception  of  tracking  error)  can  be  obtained  continuously.  In  many  systems 
where  the  operator  is  placed  in  a  supervisory  role,  very  few  overt  responses  (e.g.  button  presses)  may  be 
made  even  though  the  operator  is  engaged  in  considerable  cognitive  activity.  In  such  a  situation  the 
behavioral  measure  provides  an  impoverished  sample  of  the  mental  activity  of  the  operator. 
Psychophysiological  measures,  on  the  other  hand,  may  be  recorded  continuously  without  respect  to  overt 
responses  and  may  provide  a  measure  of  the  covert  activities  of  the  pilot.  Second,  in  some  instances, 
psychophysiological  measures  may  provide  more  information  when  coupled  with  behavioral  measures  than 
behavioral  measures  alone.  For  example,  changes  in  reaction  time  may  reflect  contributions  of  both  central 
processing  (working  memory)  and  response-related  processing  to  workload;  however,  when  coupled  with 
P300  amplitude  and  latency  changes,  such  changes  may  be  more  precisely  localized  to  central  processing 
stages  than  to  response-related  processing  (Donchin  et  al.,  1986). 

How  would  a  psychophysiologicaliy  based  adaptive  system  work?  It  is  presumed  that  pre-existing 
"profiles"  can  be  established  for  each  pilot  indicating  the  correspondences  between  a  specific  pilot  state 
(such  as  reduced  vigilance,  increased  workload,  etc)  and  the  measured  physiological  signals  (e.g.,  see 
Carriero,  1977).  A  psychophysiological  adaptive  system  would  assess  these  states  "on-line,"  feeding  this 
information  to  a  secondary  logic  system  (e.g.,  an  expert  system)  that  would  determine  whether  adaptive 
changes  are  required.  This  concept  is  not  new,  having  first  been  proposed  in  the  context  of  a  program 
funded  by  DARPA  in  the  1970's  for  a  "biocybernetic  communication  channel"  between  pilot  and  aircraft 
(Donchin,  1980;  Gomer,  1980).  Hancock  et  al.  (1985)  reported  a  more  recent  proposal  in  which 
physiological  signals  would  be  coupled  to  an  intelligent  subsystem  that  implements  adaptive  functions  such 
as  task  allocation. 

Various  candidate  psychophysiological  measures  are  available,  most  of  which  can  be  drawn  from  the 
literature  on  physiological  measures  of  workload.  Probably  the  most  widely  supported  measures  are  those 
based  on  brain  electrical  activity,  and  in  particular  the  P300  and  N100  event-related  potentials  of  the  brain 
(Donchin  et  al.,  1986;  Parasuraman,  1990).  Both  these  ERP  measures  have  been  shown  to  provide  a  fairly 
sensitive  measure  of  mental  workload  in  multi-task  situations.  In  addition,  these  brain  potentials  have  a 
measure  of  diagnosticity  as  well.  The  P300  has  been  shown  to  reflect  primarily  the  allocation  of  perceptual- 
cognitive  resources  and  not  response-related  processes  (Donchin  et  al.,  1986).  The  N100  brain  potential, 
on  the  other  hand,  has  been  found  to  reflect  attentional  resources  associated  with  early  information- 
processing  stages  (Hilfyard  &  Kutas,  1983).  Of  the  autonomic  measures,  heart  rale  variability,  and  particularly 
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in  the  0.1  Hz  frequency  band,  has  been  reported  to  be  a  sensitive  index  of  momentary  workload.  This 
measure  has  the  advantages  of  requiring  simpler  instrumentation  than  ERPs.  However,  heart  rate  variability 
may  be  less  diagnostic  a  measure  than  ERPs,  and  may  reflect  a  number  of  different  sources  contributing  to 
workload. 

Several  critical  conceptual  and  technical  issues  must  be  tackled  before  psychophysiological  adaptive 
systems  could  be  fielded.  The  criteria  of  sensitivity  and  diagnosticity  that  apply  to  behavioral  workload 
measures  apply  equally  forcefully  to  psychophysiological  measures.  In  fact  one  could  make  the  argument 
that  the  sensitivity  issue  applies  more  stringently.  This  is  because  it  is  generally  possible  to  attach  some 
meaning  to  absolute  values  of  behavioral  measures,  even  with  only  limited  knowledge  of  the  stimulus  context. 
For  example,  a  reaction  time  of  200  msec  or  an  accuracy  score  of  95%  can  be  taken  to  represent  highly 
efficient  performance.  The  meaning  of  a  P300  amplitude  of  15  pV,  on  the  other  hand,  cannot  be 
determined  without  details  of  the  experimental  and  recording  conditions.  Major  technical  problems  (e  g. 
artifact-free  recording  in  noisy  cockpit  environments;  reliable  single-trial  recordings,  etc.)  have  to  be  solved 
before  psychophysiological  measures  could  be  used  routinely  in  aircraft.  In  addition,  factors  such  as 
reliability,  cost,  and  pilot  inconvenience  and  mistrust,  must  be  dealt  with.  A  great  deal  of  technical 
development  must  be  devoted  to  psychophysiological  methodologies  before  they  could  be  seriously 
considered  for  fielding  in  airborne  adaptive  systems. 


Pilot  Performance  Models 

Pilot  performance  models  are  important  to  the  design  and  study  of  automation  issues  because  they 
provide  the  basic  framework  with  which  to  understand  pilot  behavior.  As  such,  the  adoption  of  an  appropriate 
pilot  performance  mode!  is  going  to  be  critical  to  the  successful  development  of  an  automation  system, 
because  the  assumptions  implicit  to  a  model  will  carry  through  to  affect  the  way  in  which  the  automation  system 
is  designed.  Further,  the  pilot  performance  model  is  going  to  be  even  more  critical  in  the  development  of 
adaptive  automation  systems  because  the  very  complex  interaction  between  the  pilot  and  the  automation 
system  will  change  substantially  as  a  function  of  the  level  of  automation,  and  the  shifting  levels  of  automation 
will  in  turn  be  affected  by  the  model.  The  advent  of  adaptive  automation  therefore  makes  the  issues 
associated  with  the  pilot  performance  model  more  critical,  as  they  will  impact  both  the  pilot’s  performance  as 
well  as  the  researcher/designer's  understanding  of  it.  This  section  of  the  report  will  describe  some  of  the 
current  and/or  more  promising  models  of  human  performance  which  hold  promise  in  attacking  the  problems  of 
adaptive  automation  in  the  cockpit. 
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Pilot  measurement  methods  have  some  4nherent  limitations.  They  may  be  too  sensitive  to  small  local 
fluctuations  in  pilot  workload  or  physiological  states,  measurement  technologies  may  be  costly,  impractical  or 
intrusive  (e..g,.particularly  for  physiological  measurement),  and  pilots  may  be  unwilling  to  accept  being 
"monitored*  in  the  cockpit.  An  alternative  to  measurement  is  to  model  the  performance  of  the  pilot  and  to 
base  adaptation  on  features  of  a  given  performance  model.  Several  optimal  performance  models  for  adaptive 
systems  have  been  proposed.  A  number  of  information-processing  models  of  human  cognition  are  also 
relevant,  although  these  were  not  specifically  developed  in  the  context  of  adaptive  systems.  We  consider 
three  such  categories  of  models  which  seem  to  have  the  most  promise  in  application  to  adaptive  systems. 

Intenf  Inferencino  Models.  A  major  advantage  of  the  modelling  approach  is  that  several  pilot  mclvis 
can  be  incorporated  into  the  adaptive  aiding  system  in  the  cockpit.  For  example,  a  data  base  of  tasks  and  a 
rule  base  for  assessing  workload  in  different  task  configurations  can  be  developed.  Alternatively,  a  rule  base 
linking  operator  intentions  to  specific  responses  can  also  be  used.  These  will  permit  the  adaptive  system  to 
acquire  knowledge  about  operator  actions.  This  knowledge  is  useful  for  the  computer  to  adapt  to  the 
intentions  of  the  human  operator.  Rouse,  Geddes,  and  Curry  (1986)  stated  ’hat  the  goal  of  performance 
models  is  to  allow  the  adaptive  system  to  evaluate  the  operator's  :  (1)  present  goals,  as  well  as  present  and 
future  behavior;  (2)  efficiency  in  performing  u  task;  situationa.  awareness;  (4)  available  information¬ 
processing  resources;  and  (5)  planned  goals  and  'ctions. 

Intent  inferencing  models  mar  allow  these  goals  to  be  realized.  In  this  kind  of  model,  available  pilot 
resources  and  performance  are  used  as  predi>,‘ors  to  decide  when  let  the  intelligent  aid  intervene. 
Intentions  can  be  represented  in  a  number  of  ways,  although  script-based  representations  have  been  most 
commonly  used.  Examples  of  in',  nt  inferencing  models  are  those  of  Geddes  (1985),  Greenstein  & 
Revesman  (1986),  and  Govindaraj  &  Rouse  (1981).  These  two  last  models  characterize  human  intentions  as 
responding  to  task  requirements.  That  is,  humans  could  not  have  any  goals  besides  those  imposed  on  them 
by  task  demands.  Such  an  assumption  may  be  violated  in  certain  operational  settings  (Geddes,  1985). 
Geddes  included  in  his  model  the  interpretation  of  human  actions,  the  system  state,  and  the  state  of  the  world 
in  the  context  of  scripts,  plans  and  goals.  His  model  acknowledges  the  existence  of  human  thoughts, 
personal  goals  and  consequently  human  intentions  which  are  not  specifically  related  to  the  context  of  task 
demands.  It  remains  to  be  seen  whether  script-based  representations  can  capture  all  aspects  of  pilot 
intentions  and  goals  in  the  complex  environment  of  the  fighter  aircraft. 


Optimal  (Mathematical!  Models.  Examples  of  optimal  models  include  those  based  upon  signal 
detection  theory,  information  theory,  queuing  theory,  sampling  theory,  and  control  theory.  Each  of  these 
theories  offer  conlext-freo. representations  of  normative  human  performance  in  tasks  that  can  be  related  to 
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task  components  in  various  operational  domains.  Queuing  theory  is  considered  here  as  an  example  of  such 
models.  This  model  is  chosen  from  the  many  available  models  because  its  range  of  applicability  is  quite  broad. 
The  reader  interested  in  other  mathematical  models  of  human  performance  can  refer  to  Sheridan  and  Farrell 
(1974)  and  Rouse  (1980). 

Optimal  control  models  such  as  that  of  Baron  and  Levinson  (1980)  focus  on  the  prediction  of  human 
control  dynamics  under  various  load  conditions.  Models  of  instrument  scanning  (monitoring)  based  on 
sampling  theory  (e.g.,  Moray,  1984)  have  also  been  proposed.  Pilots  are  clearly  engaged  in  both  these 
activities,  as  well  as  other  decision-making  and  planning  activities.  A  general  mathematical  model  that  can  be 
applied  to  this  multitask  situation  is  queuing  theory.  The  queuing  model  proposed  by  Walden  and  Rouse 
(1978)  was  developed  in  response  to  the  limitations  of  some  of  the  previously-mentioned  models  as  well  as 
to  meet  the  requirement  for  computer  aiding  schemes  for  pilot  multitask  performance.  Their  model  focused 
on  time  sharing  between  monitoring,  flight  control,  and  other  tasks.  The  model  considered  the  monitoring 
task  as  a  queuing  system  which  can  be  modeled  as  a  "single  server"  with  subsystem  events  called 
"customers"  and  with  the  control  task  incorporated  as  a  special  queue.  The  system,  according  to  this  model, 
works  as  follows:  Once  the  customers  arrive  at  the  control  task  queue  they  can  control  the  service  of  a 
subsystem  event.  From  what  preceded,  a  customer  in  this  case  can  be  defined  as  a  "significant  amount  of 
display  error".  Therefore,  when  a  major  error  is  displayed,  the  subsystem  service  is  preempted  and  a  control 
action  is  taken  to  eliminate  the  error. 

To  demonstrate  the  usefulness  and  workability  of  their  model,  Walden  and  Rouse  (1978)  conducted 
an  experiment  to  study  performance  in  a  simulated  multitask  flight  management  situation.  Two  independent 
variables  were  manipulated,  the  inter-arrival  times  of  customers  and  the  level  of  difficulty  of  the  flight  path  to  be 
followed  by  the  pilot.  The  goal  behind  this  study  was  to  compare  the  effects  of  automated  versus  manual 
control  on  system  performance.  Subjects  were  tested  in  two  sessions,  a  training  session  for  the  monitoring 
task,  and  a  test  session  in  which  subjects  performed  under  automation  (with  autopilot  failure).  The 
independent  variable  was  inter-arrival  times  (30, 60  and  90  sec.  per  subsystem).  Dependent  variables 
consisted  of  reaction  time  to  the  subsystem  event  and  the  time  spent  for  the  diagnosis  of  the  event.  An 
easy  terrain  map  was  given  to  subjects  for  the  first  trial  of  a  session .  The  second  trial  consisted  of  a  more 
complex  map.  A  single  control  task  was  used  in  the  first  session  so  as  to  establish  baseline  performance. 

The  rest  of  the  trials  involved  the  three  levels  of  monitoring  workload.  Dependent  variables  in  the  second 
session  involved  the  two  measures  mentioned  earlier  as  well  as  aircraft  position  and  attitude  and  finally  pilot 
control  inputs. 

The  results  from  this  study  showed  that  subjects  usually  "smoothed"  the  version  of  the  designed 
map  to  follow  a  simpler  and  more  realistic  map  by  adopting  a  "mental  smoothing  procedure"  (mental  model).  It 
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was  also  found  that  the  average  service  time  was  independent  of  the  customer  arrival  rate.  That  is,  the 
increase  in  the  subsystem  service  time  was  related  to  the  mere  presence  of  the  oontrol  task  and  not  to  its 
level  of  difficulty.  The  subsys  em  event  waiting  tine  was  also  delayed  by  the  simple  presence  of  the  control 
task.  False  alarm  rate  was  much  higher  for  the  control  task  with  the  complex  map  than  at  the  same  task  with 
the  simpler  map  or  no  control  task  at  all.  Also  there  were  more  incorrect  actions  with  no  control  task  than  with 
the  control  task  with  both  simple  and  complex  maps.  These  incorrect  actions  increased  the  average  and 
standard  deviation  of  waiting  time  for  lower  priority  processes. 

After  investigating  the  adaptability  of  the  queuing  model  of  pilot  decision-making  in  a  flight 
management  system,  Walden  and  Rouse  (1978)  provided  evidence  that  their  queuing  model  permitted 
variations  in  the  control  task  service  rate  so  as  to  provide  realistic  predictions  of  the  performance  of  the 
subsystem  monitoring  task  This  successful  application  illustrates  the  potential  use  of  this  model  in  adaptive 
automation  systems. 

These  mathematical  models  have  proved  useful  in  certain  applications.  However,  they  have  some 
limitations.  First,  they  require  complicated  computations  even  for  simple  tasks.  Second,  if  implemented  in 
an  adaptive  system,  they  require  high  expertise  to  calibrate,  change,  extend,  and  otherwise  modify  the 
software,  finally,  these  models  are  oriented  toward  abstracted  parts  of  the  task  rather  than  the  contextual 
reality  within  which  these  operations  are  actually  achieved  (Rouse  et  al,  1986). 

There  is  also  some  concern  whether  modelling  approaches  such  as  intent  inferencing  or  queuing 
theory  are  appropriate  for  a  tactical  environment.  Most  artificial  intelligence  (Al)  approaches  require  production 
systems  that  are  based  on  deductive  logic  and  require  a  well-bounded  problem.  (Al  approaches  to  decision 
making  based  on  parallel  distributed  processing  are  still  in  their  infancy.)  Barrett  &  Donnell  (1989)  and  Nau  & 
Reggia  (1986)  argue  that  most  human  problem  solving  involves  inductive  or  abductive  reasoning.  In 
particular,  situation  assessment  may  be  an  inductive  process;  and  other  real-world  decision  problems  are  too 
complex  and  too  poorly  bounded  for  current  computational  approaches  (Barrett  &  Donnell,  1989).  A  tactical 
environment  is  difficult  to  predict  because  unforeseen  tactics  are  not  only  possible,  but  likely.  To  put  it  more 
strongly,  an  adversary  will  do  all  it  can  (deception,  new  technology,  etc)  to  create  a  non-stationary, 
unpredictable  probabilistic  environment.  Barrett  and  Donell's  (1989)  argument  is  that  in  such  highly  non¬ 
linear  situations  human  reasoning  skills  are  more  veridical  than  expert  systems  that  depend  upon  deductive 
approaches. 

This  is  no*  an  argument  against  all  modelling  approaches.  Barrett  &  Donnell  (1989)  proposed  an 
architecture  to  combine  human  reasoning  and  machine  computational  abilities.  Their  approach  to  decision 
aiding  utilizes  the  distinction  between  inductive  and  deductive  reasoning  as  a  basis  for  allocating  functions 
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between  the  operator  and  machine  components  of  a  system. 7  Others  have  developed  computational  aiding 
systems  based  on  abductive  approaches  wherein  evidence  implies  a  reduced  subset  of  possible  hypotheses 
(Nau  &  Reggia,  1986;  Josephson  et  at.,  1987).  These  modelling  approaches  are  also  congruent  with  the 
critical-event  logic  adaptive  systems  described  above.  A  system  driven  by  critical  events  leaves  most  ot  the 
situation  assessment  and  high-level  tactical  reasoning  to  the  human  whereas  the  adaptive  system  performs  in 
situations  where  either  reaction  time  or  complex  computational  requirements  demand  an  automatic  response. 

Wickens’  Multiple-Resource  Theory.  This  theory  grew  out  of  considerations  of  the  pattern  of 
interference  obtained  in  dual-task  performance  (Wickens,  1979, 1984).  The  theory  postulates  a  limited  set  of 
information-processing  structures  or  resources  that  jointly  determine  the  degree  to  which  performance 
interference  occurs.  That  is,  if  two  tasks  compete  for  the  same  human  information  processing  resource,  and 
their  combined  resource  requirements  are  much  higher  than  the  available  resources,  then  interference  will 
occur.  The  resources  that  Wickens  (1984)  considered  included  stages  of  processing  (central  processing 
versus  response  processing),  modalities  of  input  (visual/auditory)  and  output  (manual/verbal),  and  type  of 
coding  in  memory  (verbal/spatial).  The  model  has  been  very  influential  in  engineering  psychology  and  has 
stimulated  a  voluminous  amount  of  research. 

The  multiple-resource  model  has  several  implications  for  time-sharing  performance  in  a  multitask 
environment  (Wickens,  1984).  First,  if  two  (or  more)  tasks  require  the  use  of  different  resources  (e.g., 
verbal/spatial  codes)  they  will  be  time-shared  efficiently.  Second,  if  the  two  tasks  share  the  same  resources 
then  there  will  be  conflict  in  time-sharing  the  two  operations  and  performance  may  suffer  if  the  resource 
demand  is  high.  Third,  an  increase  in  task  difficulty  will  automatically  cause  an  increase  in  the  task  demand  in 
resources.  And  if  some  of  these  resources  are  needed  for  a  concurrent  task  then  the  performance  of  the 
latter  will  be  affected.  In  the  case  where  those  resources  or  part  of  them  are  not  needed,  then  concurrent 
task  performance  will  remain  stable.  Finally,  if  task  difficulty  is  increased  still  further  so  that  there  is  resource 
scarcity  due  to  division  between  two  tasks,  then  performance  on  each  task  will  reflect  the  operators  priorities. 

The  implications  of  this  model  for  adaptive  systems  are  fairly  straightforward.  This  model  helps 
allocate  tasks  that  are  not  competing  for  the  same  resources.  If  competition  takes  place,  the  model  will  serve 
as  a  tool  to  assess  the  impact  of  that  competition  in  order  to  appropriately  allocate  tasks  to  the  computer  and 
operator.  The  model  offers  information  on  how  to  solve  the  task  allocation  problem  by  distinguishing 


7  This  approach  has  been  applied  as  a  model  in  the  Knowledgeable  Operator  Analysis-  Linked  System 
(KOALAS)  for  decision  aiding  and  has  in  turn  been  applied  to  US  Navy  Command  and  Control  (C2)  in  an 
airborne  electronic  countermeasures  system  (AEW-  RTAS)  (Barrett,  1988;  Stokes  &  Barrett,  1988a; 
Stokes  &  Barrett,  1988b). 
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between  concurrent  tasks  and  non-concurrent  tasks.  Evaluating  the  effect  of  competition  between  two  tasks 
allows  a  better  distinction  of  tasks  between  the  pilot  and  the  automation.  In  other  words,  the  model  allows 
the  designer  to  decide  when  to  automate  a  certain  task  or  when  to  allow  the  human  operator  to  carry  out 
more  than  one  task  without  affecting  system  performance  adversely.  Accordingly,  a  task  is  automated  if  its 
combination  with  a  concurrent  one  would  lead  to  a  total  demand  on  a  specific  resource  that  exceeds 
availability.  Because  the  models  predicts  a  distinction  between  a  competitive  task  and  a  non-competitive 
task  the  system  designer  can  design  the  adaptive  logic  to  ensure  that  time-shared  tasks  do  not  compete  for 
the  same  resources  and  hence  degrade  performance. 

Central  Executive  Models.  This  type  of  model  also  analyses  multitask  performance  by  referring  to 
information-processing  structures  and  their  associated  resources.  In  contrast  with  Wickens’  theory, 
however,  this  class  of  model  assumes  the  existence  of  a  central  executive  process  which  is  modality  free  and 
supervises  other  subsystems  under  its  control.  Two  such  models  will  be  discussed,  Baddeley’s  (1986) 
working  memory  model,  and  Shallice's  (1982)  supervisory  attention  system  model. 

Baddeley  (1986)  developed  his  working  memory  model  as  a  response  to  the  "traditional"  multi-store 
(e.g.,  short-term  and  long-term  memory  stores)  models  of  memory.  The  theory,  as  originally  articulated  by 
Baddeley  and  Hitch  (1974),  argued  that  performance  on  a  range  of  cognitive  tasks  required  a  "working 
memory",  i.e  a  temporarily  active  part  of  the  memory  system  that  was  devoted  to  carrying  out  and  storing 
partial  computations  required  to  execute  a  given  cognitive  task,  such  as  reasoning  or  reading.  In  that  sense 
working  memory  was  not  unique  to  so-called  short-term  memory  tasks  but  was  a  general  characteristic  of  many 
cognitive  tasks. 

On  the  basis  of  a  review  of  existing  memory  research  and  his  own  studies,  Baddeley  (1986) 
postulated  the  need  for  a  multicomponent  working  memory  system.  He  proposed  that  working  memory 
comprised  a  central  executive  that  coordinated  and  directed  the  operation  of  two  "slave"  subsystems,  the 
articulatory  loop  and  the  visuo-spatial  "scratchpad".  These  two  subsystems  are  respectively  the  working 
memory  systems  dealing  with  verbal  and  nonverbal  processing  respectively.  In  this  respect,  Baddeley's 
model  is  also  a  multiple-resource  one  like  Wickens’.  However,  it  differs  from  Wickens'  model  in  postulating  the 
need  for  a  central,  modality-free  executive  that  oversees  the  operation  of  the  two  subsystems. 

The  articulatory  loop  subsystem  is  similar  to  a  closed  loop  on  a  tape  reoorder.  The  loop  contains  a 
set  of  articulatory  programs  which  are  used  to  achieve  the  process  of  articulation,  thus  preventing  the  trace  of 
the  articulated  stimulus  from  fading.  The  notion  of  stronger  articulation  leading  to  better  recall  fits  nicely  with 
this  model.  Moreover,  findings  about  the  impact  of  articulatory  suppression  also  support  the  existence  of 
this  subsystem.  The  second  postulated  subsystem,  the  visuo-spatial  scratchpad,  deals  with  the 
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processing  of  visuo-spatial  material.  A  general  finding  in  the  past  has  been  that  processing  of  sentences  with 
large  amounts  of  visual  imagery  are  disrupted  by  visual  processing,  whereas,  non-imageable  or  abstract 
phrases  are  disrupted  by  auditory  processing.  Baddeley  and  his  colleagues  found  that  the  disruption  of  the 
operation  of  the  scratchpad  by  a  competitive  tracking  task  affected  long-term  learning  of  manipulating  visuo- 
spatial  images.  These  results  suggest  the  existence  of  a  temporary  visuo-spatial  store  which  can  retain  and 
manipulate  images  and  which  is  very  sensitive  to  disruption  by  concurrent  spatial  processing.  Finally,  the 
central  executive  is  considered  modality  free  since  it  is  supposed  to  relate  between  different  subsystems 
which  use  specific  modalities.  Thus,  the  central  executive  has  attentional  capacities  and  the  ability  to  select 
and  operate  control  processes. 

Shallice  (1982)  postulated  a  central  executive  model  of  human  cognition  that  has  similarities  to 
Baddeley's  model.  He  bases  his  executive  control  system  on  elementary  units  called  schemas.  Schemas 
are  highly  specialized  routine  programs  that  control  a  particular  automatic  (overieamed)  action  or  skill  such  as 
opening  a  door  or  eating.  These  programs  are  action  schemas,  in  that  once  triggered  by  a  given  appropriate 
stimulus,  they  "run  off  processing  operations  automatically  to  lead  to  the  desired  and  appropriate  response. 
Shallice  assumes  that  many  schemas  can  be  activated  simultaneously  at  any  given  time  because  they  are 
completely  independent  of  each  other.  Most  actions  are  controlled  by  schemata  that  are  given  the 
appropriate  triggering.  Schemata  can  operate  at  different  levels  from  a  simple  conscious  behavior  to  an 
overlearned  one  to  a  more  complex  form. 

Thus  far  this  model  is  not  markedly  different  from  other  theories  of  automatic  and  controlled  or 
effortful  processing  (Hasher  &  Zacks,  1979;  Schneider  &  Shiffrin,  1977).  The  new  feature  is  that  the  process 
of  selecting  the  set  of  schemas  to  be  run  is  postulated  to  involve  two  different  processes  (the  distinction  is 
qualitative):  (1)  contention  scheduling ,  which  is  found  in  both  automatic  and  controlled  selection  of  action 
schemas,  or  in  Shallice's  words  "routine  and  non  routine  selection";  and  (2)  the  supervisory  attentional 
system  (SAS),  which  is  involved  only  in  controlled,  effortful  selection.  In  contention  scheduling,  selection  is 
achieved  according  to  priorities  and  environmental  cues.  Such  a  process  is  labeled  ‘automatic  conflict 
resolution"  because  it  ensures  the  efficient  use  of  different  processing  operations;  thus  avoiding  probable 
conflict  between  different  schemata  (e.g.,  the  use  of  the  same  response  effector  to  two  simultaneous  inputs). 
The  selection  of  a  schema  is  ensured  by  mutual  inhibition  between  different  units.  This  inhibition  needs  to 
be  maintained  until  the  selected  schema  action  is  completed.  Also  schemas  need  to  be  maintained  for 
extended  periods  of  time  so  they  can  be  used  if  appropriate  environmental  stimuli  occur.  In  addition  to  this 
conflict  resolution  process,  there  is  an  overall  controller:  the  SAS,  which  enables  operations  on  schemas  in 
every  domain.  In  contrast  to  the  contention  scheduling  mechanism,  which  is  fast,  routine,  and  inflexible, 
selection  based  on  the  SAS  is  slow,  non  routine,  and  flexible.  The  SAS  biases  the  activation  of  schemaia 
by  favoring  one  schema  over  the  others.  It  is  a  conscious  attentional  means  of  control.  This  selective 
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process  is  limited  in  capacity  and  is  generally  invoked  in:  (1)  tasks  that  are  involved  in  planning  or  decision 
making:  (2)  situations  in  which  the  automatic  processes  appear  to  be  running  into  difficulties;  (3)  where  novel 
or  poorly  learned  sequences  of  acts  are  involved;  or  (4)  where  the  situation  is  judged  to  be  dangerous  or 
technically  difficult. 

All  of  these  conditions  are  likely  to  be  ones  in  which  automation  may  be  invoked  in  the  cockpit.  The 
implications  of  this  model  for  adaptive  systems  differ  somewhat  from  those  of  the  related  resource  models. 

As  mentioned  earlier,  multiple  resource  theory  (Wickens,  1979)  predicts  a  deterioration  in  performance  if  two 
or  more  tasks  compete  for  the  same  information  processing  resources.  An  adaptive  system  consistent  with 
this  model  would  invoke  actions  when  the  demands  of  competing  tasks  exceed  the  available  resources. 

On  the  other  hand,  central  executive  models  (Shallice,  1980;  Baddeley,  1986)  postulate  that  performance 
deterioration  is  not  inevitable.  If  an  operator  needs  to  perform  two  concurrent  actions,  both  of  which  are 
resource  demanding,  then  the  appropriate  way  to  achieve  it  is  to  create  a  higher-order  schema  that  oversee 
both  sequences  (Shallice,  1980).  This  may  be  possible  through  training  or  skill.  In  the  case  that  both 
actions  can  be  performed  automatically  without  the  need  for  the  SAS,  then  they  will  be  automatically 
synchronized  and  hence  not  need  intervention  from  the  adaptive  system.  Hence,  resource  demanding 
tasks  (which  require  attentional  control)  must  either  be  performed  with  synchrony  or  in  alternation.  Moreover, 
newly  learned  actions  cannot  be  automatically  performed  because  their  schemas  are  small,  relatively 
specialized  sub-actions.  Also,  their  triggering  conditions  are  not  compatible  with  the  actual  conditions  that 
occur.  In  this  case  continual  attentional  monitoring  is  required  and  selection  is  delayed  by  deliberate 
attentional  activation.  On  the  other  hand,  well-learned  actions  are  well  defined  and  their  schemas  are 
relatively  large,  organized  units  of  behavior.  Their  triggering  conditions  are  very  compatible  to  the  current 
conditions.  Once  the  schemas  of  well-learned  actions  have  been  selected,  they  can  maintain  control  for  long 
periods  of  time. 

Based  on  the  above  mentioned  points,  the  implications  of  this  category  of  model  for  an  intelligent  aid 
can  be  defined  as  follows:  1)  The  model  provides  an  assessment  of  the  task  sequences  needing  attentional 
control  and  those  that  do  not  need  this  type  of  control.  Adaptation  can  thus  be  based  on  this  requirement. 

2)  By  determining  when  conditions  permit  or  do  not  permit  the  activation  of  the  supervisory  attentional 
system,  the  model  shows  which  tasks  are  easier  to  be  earned  out  by  the  pilot  or  by  the  automaton.  3)  The 
model  can  be  used  to  assess  when  the  human  operator  can  carry  out  more  than  one  task  without  harming 
overall  performance.  Thus,  it  could  be  used  to  decide  the  appropriate  time  to  allocate  certain  tasks  to  the 
pilot  or  to  the  automation.  This  aspect  of  the  model  does  not  differ  from  other  resource  models,  except  that 
the  predictions  of  task  interference  differ.  4)  Shallice's  model  may  be  particularly  well  suited  to  test  whether 
adaptive  actions  have  adverse  transient  or  tong-term  effects  on  performance.  Such  effects  may  take  the 
form  of  a  deterioration  in  performance  following  a  transition  from  automated  to  manual  mode,  even  though 
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resource  models  predict  no  degradation  in  performance.  If  the  reversion  to  manual  performance  occurs  after 
a  very  long  period,  the  appropriate  schema  for  multitask  scheduling  and  coordination  may  need  some  time  to 
be  retrieved,  "re-compiled"  and  activated.  Thus  performance  may  suffer.  Alternatively,  adaptive  logic  that 
rapidly  cycles  the  pilot  between  automated  and  manual  modes  may  also  degrade  performance  because  the 
appropriate  schemata  are  not  properly  compiled  in  the  first  place.  Research  is  currently  ongoing  to  test  these 
ideas.  If  support  for  such  effects  is  found,  the  implication  for  adaptive  algorithms  will  be  that  such  adaptive 
methods  will  have  to  take  into  account  time  periods  between  cycles  of  automation  and  manual  performance 
and  adjust  task  allocation  acoordingly. 


Training  Issues 

Successful  training  of  pilots  to  use  adaptive  systems  will  be  essential  to  the  effectiveness  of  such 
systems.  As  discussed  previously,  automation  can  place  conflicting  demands  upon  pilots  which  they  may  not 
be  well-equipped  to  meet  (e.g.  passive  monitoring  versus  active  control)  unless  they  have  been  specifically 
trained  to  cope  with  these  demands.  However,  training  issues  in  automated  systems  have  not  received 
much  attention,  and  in  fact  there  is  virtually  no  literature  comparing  different  training  methods  for  users  of 
adaptive  systems. 

It  was  suggested  earlier  that  inadequate  training  may  lead  to  several  automation-induced  problems  in 
the  cockpit.  For  example,  the  negative  effect  of  automation  on  monitoring  performance  may  be  related,  in 
part,  to  a  lack  of  “automation-based"  skills.  This  reflects  inappropriate  training  because  automation 
necessitates  a  shift  from  psycho-motor  skills  to  more  cognitive  and  problem-solving  skills,  which  may  not  be 
emphasized  in  the  training  program  (Idaszak,  1989).  Most  training  programs  are  based  on  eliciting 
appropriate  responses  from  the  pilot  to  display  messages  and  flight  conditions.  Pilots  are  taught  how  to 
respond  to  a  particular  signal  in  a  specific  situation  but,  learning  information-processing  skills  is  not  typically 
emphasized  (Braune  and  Trollip,  1981). 

Adaptive  automation  suggests  the  relevancy  of  adaptive  training  methods  already  in  use.  In  contrast 
to  adaptive  automation  research,  there  is  a  long  history  of  research  on  adaptive  training  methods.  It  had  long 
been  held  that  adaptive  training  is  a  superior  form  of  training,  and  many  investigators  advocated  its  feasibility. 
However,  as  Lintern  and  Gopher  (1978)  showed  in  a  comprehensive  literature  review ,  the  empirical  evidence 
for  its  effectiveness  was  quite  weak. 

Adaptive  training  has  generally  been  contrasted  to  more  traditional  part-task  training.  Adaptive 
training  is  the  gradual  increase  in  the  difficulty  and  complexity  of  the  training  task.  At  first,  a  complex  task 
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made  up  of  several  component  tasks  is  made  easy  (by  manipulating  task  component  parameters)  and 
presented  at  a  slow  rate,  permitting  the  operator  to  quickly  gain  mastery.  As  efficiency  increases,  task 
difficulty  and  presentation  speed  are  increased  adaptively  to  keep  performance  levels  roughly  constant.  On 
the  other  hand,  part-task  training  is  the  presentation  of  different  parts  of  the  task  individually.  As  mastery 
over  the  individual  parts  is  gained,  the  whole  task  is  gradually  introduced.  Research  has  focused  on 
determining  which  of  these  training  strategies  is  the  more  efficient. 

it  has  been  always  argued  that  adaptive  training  has  more  advantages  for  learning  over  other  methods 
and  strategies.  This  claim  is  based  on  several  assumptions:  (1 )  learning  ^ny  complex  perceptual-motor  skill 
has  to  be  time-shared  with  its  execution,  which  can  interfere  with  learning  (Klein  &  Posner,  1974);  (2) 
decreasing  the  speed  of  presentation  of  the  task  components  reduces  achievement  demands,  which  gives 
the  subject  more  time  to  learn  the  task  well  (Gopher  &  North,  1977);  and  (3)  the  nature  and  the  relationship  of 
the  task  components  when  presented  at  a  slow  rate  are  the  same  as  in  the  high-rate  complex  task. 

In  a  study  by  Mane.  Adams  and  Donchin  (1989)  the  two  training  methods  (adaptive  and  part  training) 
were  evaluated  in  a  complex  perceptual-motor  task,  referred  to  as  "the  Space  Fortress".  In  the  pari  training 
method,  a  subject  was  trained  on  important  subtasks  before  he  was  presented  with  the  whole  task.  In  the 
adaptive  strategy  case,  a  subject  was  presented  with  the  whole  task  where  its  time  pressure  was  gradually 
corrected  to  the  subject's  performance  until  he  reached  a  level  where  he  could  deal  with  the  real-world  task 
effectively.  The  results  showed  that  part  faining  was  a  more  effective  strategy  than  adaptive  training 
While  compared  to  the  control  group  (whole  training),  the  part  training  group  was  clearly  learning  and 
performing  better  throughout  the  entire  period  of  training.  In  the  adaptive  training  situation,  a  negative 
transfer  from  slow  to  fast  speed  was  found.  This  finding  provides  evidence  that  being  trained  in  the  slow 
speed  mode,  subjects  found  it  difficult  to  adjust  to  the  the  high  speed  task.  Such  results  confirm  Lintern  and 
Gopher's  (1978)  claim  that  adaptive  training  cannot  be  effective  unless  the  transfer  is  substantial.  That  is, 
transfer  has  to  be  larger  than  the  skills  acquired  by  training  during  the  difficult  version  (or  fast  speed  in  the 
present  case).  Another  explanation  for  the  present  finding  resides  in  the  violation  of  the  third  assumption 
described  earlier;  since  the  relationship  among  the  components  of  the  task  do  not  seem  to  be  the  same  in  the 
slow  speed  case  as  in  the  fast  speed  condition. 

These  results  suggest  that  adaptive  training  methods  alone  may  not  be  the  most  appropriate 
technique  to  help  train  users  of  automated  systems,  although  they  may  still  be  helpful  when  combined  with 
other  methods.  This  is  a  somewhat  unfortunate  state  of  affairs,  because  the  two  concepts,  adaptive 
automation  and  adaptive  training,  are  related,  and  it  would  be  parsimonious  if  a  single  unifying  principle, 
could  be  applied  both  to  the  design  of  the  automation  and  to  the  design  of  the  training  program  for  the  users 
of  automation  for  adapting  to  both  his  needs  and  abilities.  Development  of  effective  training  techniques  for 
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adaptive  automation  is  a  research  priority  because  the  shifting  task  requirements  implicit  in  adaptive 
automation  will  require  diagnostic  and  control  skills  substantially  different  from  those  of  current,  fixed 
automation  aircraft.  Unless  these  are  developed,  the  anticipated  benefits  of  adaptive  systems  may  not  be 
realized. 


Preliminary  Design  Issues  and  Principles 

Automation  has  traditionally  been  implemented  using  a  technology-centered  approach.  In  other 
words,  as  the  technology  has  become  available,  automation  has  been  introduced.  The  pitfalls  of  this  design 
approach  have  been  noted  by  many  authors  (Boehm-Davis  et  al.,  1983;  Normal  et  al.,  1988;  Wiener,  1988). 

In  general,  technology-centered  design  has  had  the  effect  of  creating  a  cockpit  environment  that  is  not  always 
fully  compatible  with  the  pilot's  needs,  capabilities,  and  limitations.  Moreover,  it  has  had  the  effect  of 
removing  the  pilot  from  meaningful  control  of  flight  operations  and  relegating  him  to  the  "peripheral"  role  of  a 
system  monitor,  at  least  in  civilian  automated  aircraft.  Numerous  other  examples  of  problems  that  this  design 
philosophy  can  cause  can  be  drawn,  not  just  from  the  domain  of  aviation,  but  also  from  other  areas  such  as 
process  control  and  manufacturing. 

The  challenge  for  human  factors  professionals  is  to  be  able  to  propose  viable  design  alternatives  to 
technology-centered  design.  In  other  words,  the  time  has  come  for  going  beyond  simply  pointing  out  the 
shortcomings  of  "traditional"  approaches  to  automation  (as  indeed  some  of  the  previous  sections  of  this 
paper  have  done!).  What  is  required  is  a  design  philosophy  that  translates  the  "lessons  learned"  from  past 
attempts  at  introducing  automation  into  design  guidelines  for  automation  that  can  be  used  effectively  by  the 
systems  designer  and  human  operator.  This  will  not  be  an  easy  task.  However,  a  preliminary  outline  of 
such  a  human-centered  approach  to  the  design  of  adaptive  automation  can  be  sketched  out  here.  The 
design  issues  and  principles  forming  part  ot  this  approach  must  remain  preliminary  because  of  the  paucity  of 
empirical  evidence  concerning  the  effects  of  adaptive  automation  on  pilot  and  system  performance. 

To  begin,  a  human-centered  approach  to  adaptive  automation  design  should  conform  to  the 
following  general  principles: 

•  The  sophistication  of  the  adaptive  automation  person-machine  interlace  will  make  the  use 

of  a  "systems  approach"  imperative.  Particularly  critical  will  be: 

User  control  and  information  requirements  are  defined  as  completely  as  possible 
without  regard  to  the  capabilities  of  the  technology. 
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General  function  of  the  automated  system(s)  are  identified  early  in  the  design 
effort:  particularly  with  regard  to  the  operator-machine  interface. 

The  technology  of  the  automation  system  must  be  fully  identified,  along  with  its 
capabilities  and  constraints,  before  functional  allocations  are  assigned  to  the 
operator  or  the  automation  system. 

-  Assumptions  regarding  the  capabilities  and  function  of  the  automation  system 
must  be  explicitly  defined  in  the  course  of  the  design  effort.  These 
assumptions  will  serve  to  drive  all  aspects  of  the  system  design  effort,  how 
operators  utilize  the  system,  and  how  the  operators  are  trained  to  use  the 
system. 

Mission  design  narratives  and  scenarios  appropriate  for  use  in  the  assessment  of 
the  complete  operator-machine  interface  are  developed  and  utilized  throughout 
the  design  effort  to  validate  and  assess  the  design  of  the  automation  system. 
Functional  analyses  are  performed  for  all  aspects  of  the  pilot  decision-making 
process.  Hierarchical  functional  flow  diagrams  are  then  generated  to  show  the 
complete  decision-making  process. 

Identify  functional  iec  irements  and  perform  function  allocation  to  the  operator 
and  automatic.!  tyotem  given  the  operator's  information  and  control 
requirements. 

Information  and  control  requirements  will  need  to  be  developed  in  an  iterative 
fashion  along  with  the  development  of  the  system  function  allocation 
throughout  the  system  design  process. 

•  The  successful  design  of  an  adaptive  automation  system  will  depend  on  a  clear  and  explicit 
statement  of  the  system  s  design  objectives.  These  objectives  will  have  to  address  a 
variety  of  factors  not  typically  found  in  the  design  of  more  traditional  aircraft  systems. 

A  functional  taxonomy  (as  opposed  to  task  taxonomy)  may  prove  necessary  in 
determining  what  decisions  in  the  autor,  ation  system  are  best  made  by  the  pilot, 
and  which  are  best  made  by  the  automated  decision  making  system.  In  this 
way,  a  person-machine  system  incorporating  adaptive  automation  will  fully  exploit 
the  unique  capabilities  of  human  intelligence  and  not  rely  solely  on  machine 
intelligence  (expert  systems).  Important  capabilities  include  the  ability  to 
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recognize  unusual  configurations  (cf.  expert  systems),  learning  from  errors, 
adapting  to  partial  information, etc.  8 

The  success  of  an  adaptive  automation  system  will  probably  depend  on  the 
explicit  identification  of  the  roles  for  the  human  operators)  of  the  system  and  the 
automation  system,  as  well  as  explicit  rules  of  conduct  as  to  how  they  should 
interact.  Adherence  to  these  rules  and  roles  would  allow  the  pilot  to 
comprehend  and  anticipate  the  shifting  of  function  allocations  by  the  adaptive 
automation  systems.  In  effect,  through  considering  the  system  design  from  a 
person  centered  (psychological)  perspective,  the  system  will  fulfill  its  stated 
operational  goals,  and  would  not  violate  its  basic  assumptions  and  objective.  In 
essence,  adaptive  automation  systems  must  incorporate  a  philosophy  regarding 
the  relationship  of  the  pilot  and  the  automation.  This  philosophy  must  be 
defined  early  in  the  design  process  and  adhered  to  throughout. 

Automation  systems  must  be  designed  to  accommodate  the  needs  of  their 
users,  and  not  the  other  way  around,  if  the  operators  are  going  to  actually  use 
the  system. 

-  The  development  of  an  adaptive  automation  system  will  necessitate  the 

establishment  of  performance  criteria  in  concert  with  the  development  of  design 
objectives.  These  criteria  will  then  be  utilized  throughout  the  system 
development  process  as  a  metric  of  the  systems  (prospects  for)  success.  In  this 
way,  it  may  be  ensured  that  the  mission  effectiveness  resulting  from  using 
adaptive  automation  be  equal  to,  or  greater  than,  mission  effectiveness  without 
the  adaptive  automation. 

•  A  probable  design  goal  in  the  development  of  an  adaptive  automation  system  will  be  to 
optimize  pilot  workload,  rather  than  simply  minimize  it. 


®Exceptions  to  this  rule  are  possible  only  in  certain  tactical  situations  (i.e.  when  an  immediate  life-and- 
death  threat  is  involved  and  there  is  sufficient  time  for  the  system  to  get  pilot  acceptance).  However,  such 
exceptions  should  be  restricted  to  such  instances  and  hence  should  be  relatively  infrequent. 
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•  In  general,  automation  systems  must  be  designed  so  that  the  pitot  remains  “in  charge"  of 
the  automation  and  remains  constantly  aware  of  its  activities.  All  efforts  should  be  made  to 
require  the  pilot's  consent  to  any  actions  that  the  automation  system  undertakes.9 

I 

•  The  long-term  impact  of  automation  on  pitot  skills  and  performance  must  be  considered 
early  in  the  design  process  and  considered  throughout  the  system  design  process. 

The  impact  of  long-term  experience  on  the  development  and/or  decrement  of 
pitot  manual  control  and  decision  making  skills,  must  be  examined  and 
understood  in  the  context  of  the  person-machine  system's  mission. 

Effective  training  programs  must  be  designed  to  optimize  the  pitot’s  contribution 
to  mission  effectiveness  in  the  aircraft  adaptive  automation  system. 

Much  conceptual  analysis  and  empirical  research  is  required  before  these  general  principles  can  be 
translated  into  emergent  design  principles  that  can  be  used  by  system  designers.  A  more  complete 
treatment  of  this  area  must  therefore  await  the  outcome  of  ongoing  research  in  a  number  of  laboratories  that  is 
aimed  at  investigating  various  aspects  of  the  effects  of  adaptive  automation  on  pilot  and  system  performance. 
What  is  clear,  however,  is  this  research  will  serve  primarily  to  fill  in  the  details  of  answers  to  specific  design 
questions.  The  framing  of  the  high-level  design  questions  can  already  be  started  (and  some  high-level 
answers  provided),  and  we  have  presented  some  preliminary  ones  below.  While  further  work  is  required  to 
categorize  these  design  questions  into  a  coherent,  unified  framework,  in  general  the  framing  of  these  design 
questions  stems  from  a  human-centered  philosophy  of  automation. 

Implementation  of  adaptive  systems  in  advanced  cockpits  requires  analysis  of  a  number  of  design 
questions.  The  questions  that  are  asked  (and  those  that  are  not)  implicitly  define  the  design  philosophy. 
Some  preliminary  questions  that  illustrate  some  of  the  issues  and  principles  discussed  in  previous  sections  of 
this  report  are  presented  below.  These  are  presented  in  a  hierarchical  manner,  such  that  answers  to  top-level 
questions  feed  into  tower-level  questions.  Component  issues  and  potential  solutions  are  given  in  more  detail 
in  Table  1. 

•  Should  the  automation  be  "static"  or  adaptive? 

9  While  parallel  distributed  processing  (i.e.  "neural  network")  systems  have  some  of  these  characteristics, 
they  have  been  primarily  applied  to  pattern  recognition  problems  and  have  yet  to  be  shown  to  be  capable 
of  handling  complex  decision-making  and  problem  solving. 
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•  Who  does  the  adapting? 

•  When  is  adaptation  invoked? 

•  How  is  adaptation  invoked? 

•  What  measurement  technology  should  be  used? 

•  What  logic  should  measurement-based  adaptive  systems  use? 

•  What  logic  should  modelling-based  adaptive  systems  use? 

•  How  is  invocation  of  adaptation  communicated  to  the  pilot? 

•  Should  "cycles"  of  automation  be  regulated  in  time? 

The  trends  in  aircraft  design  call  for  increasing  complexity,  therefore  making  the  system  management 
demands  on  the  pilot-vehicle  interface  increasingly  complex  as  well.  Thus,  the  need  for  automation  in  the 
pilot-vehicle  interlace  becomes  a  necessity.  Automation  holds  the  promise  of  reducing  much  of  the  pilot's 
task  complexity,  however,  it  also  holds  much  of  the  responsibility  for  creating  it.  If  automation  is  to  fulfill  its 
promise,  it  is  important  that  we  carefully  study,  and  come  to  understand,  how  it  impacts  upon  both  the  pilot  and 
overall  system  performance.  We,  as  system  designers,  must  be  careful  not  to  exaggerate  claims  about 
automation  until  further  empirical  evidence  is  available.  We  have  already  seen  that  the  benefits  of  existing 
cockpit  automation  has  been  tempered  by  real-world  problems,  and  the  concerns  expressed  by  the  users  of 
these  systems.  We  must  constantly  ask:  Will  automated  systems  actually  increase  workload?  What  happens 
to  operator  skill  levels  as  we  increase  the  levels  of  automation?  How  does  the  introduction  of  increasingly 
sophisticated  automation  change  the  nature  of  the  tasks  being  performed  by  the  systems'  human  operators? 

It  is  inappropriate  to  incorporate  new  automation  technologies  into  a  system  as  complex  as  the 
modem,  and  future  cockpit,  until  we  have  come  to  understand  how  this  will  impact  all  aspects  of  system 
operation.  It  is  unrealistic  to  expect  pilots  to  accept  this  technology  and  to  utilize  it  to  its  full  potential  until  they 
can  understand  it,  its  capabilities  and  limitations,  and  may  be  assured  of  its  reliability  and  safety.  To  this  end, 
as  system  designers  and  researchers,  we  must  accept  the  responsibility  for  adapting  an  adaptive  automation 
system  to  its  end  users.  We  must  consider  the  individual  needs  of  each  user,  and  accommodate  these  needs 
through  system  design  and  training  when  necessary  and  possible. 

The  successful  resolution  of  the  problems  and  concerns  represented  by  the  application  of  adaptive 
automation  technology,  will  require  a  multi-disciplinary  approach.  Specialists  from  such  areas  as  human- 
computer  interaction,  interface  design,  and  training,  will  be  needed  on  the  design  team  so  that  an  integrated 
systems  approach  may  be  employed.  We  ask  the  reader  to  remember  that  the  preliminary  design  issues  and 
principles  offered  here,  are  prospective  and  evolutionary  in  nature.  As  additional  information  is  gathered  and 
refined,  it  is  anticipated  that  potential  solutions  ( and  additional  questions)  will  be  raised  and  discussed  in  the 
context  of  the  next  generation  combat  aircraft. 
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TABLE  1 

PRELIMINARY  DESIGN  ISSUES  AND  PRINCIPLES 
FOR  ADAPTIVE  AUTOMATION  IN  AVIATION  SYSTEMS 


1.  SHOULD  AUTOMATION  BE  -STATIC"  OR  ADAPTIVE? 

1.1  Static  Automation 

1 .2  Adaptive  Automation  V 

2.  WHO  DOES  THE  ADAPTING? 

2.1  System  Designer 

2.2  Pilot 

2.3  C3I  System 

2.4  Adaptive  System  V 

2.5  Adaptive  System  with  Pilot  Input  V 

3.  WHEN  IS  ADAPTATION  INVOKED? 

3.1  Off-line 

3.1.1  Mission  Phase 

3.1.2  Pre-programmed  Mission  Phase 
3.4  On-line 

3.4.1  Critical  Events  (Programmed  lor  Off-line) 

3.4.2  On-line  (At  times  determined  by  Pilot  Measurement  or  Modelling)^ 
3.4  Combined  Off-line/On-line  V 

4.  HOW  IS  ADAPTATION  INVOKED? 

4.1  Critical-Event  Logic 

4.2  Pilot  Measurement 

4.2.1  Static  Workload  Assessment 

4.2.2  Dynamic  Workload  Assessment  V 

4.2.3  Psychophysiological  Assessment 

4.3  Piiot  Performance  Modelling  V 

4.3.1  Optimal  Performance  (Mathematical)  Modelling 

4. 3.1.1  Queuing  Theory 

4.3. 1.2  Sampling  and  Supervisory  Control  Theory 

4.3.2  Human  Information  Processing  Modelling 

4.3.2. 1  Multiple  Resource  Theory 

4.3. 2. 2  Central  Executive  Models 

4.3.2. 3  Inference  (Intention)  Modelling. 

5.  WHAT  MEASUREMENT  TECHNOLOGY  SHOULD  BE  USED? 

5.1  Primary-task 

5.2  Secondary-task 

5.2.1  Probe  reaction  time 

5.2.2  Sternberg  task 

5.2.3  Random  number  generation 
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5.3  Psychophysiological 

5.3.1  ERPs  (P300,  N100) 

5.3.2  EEG 

5.3.3  Heart  rate  variability 

5.4  Subjective 

5.4.1  NASA-TLX 

5.4.2  SWAT 

5.4.3  OWLKNEST 

5.5  Combined 

5.5.1.  WCFIELDE 

6.  WHAT  ADAPTIVE  LOGIC  SHOULD  MEASUREMENT-BASED  ADAPTIVE  SYSTEMS 
USE? 

6.1  Simple  Workload  Threshold  (Workload  "RedLine") 

6.2  Dynamic  Workload  Threshold  (Recent  Workload  History) 

6.3  Workload  Profile 

6.4  Psychophysiological  Activity  Profile 

7.  WHAT  ADAPTIVE  LOGIC  SHOULD  MODELLING-BASED  ADAPTIVE  SYSTEMS  USE? 

7.1  Optimal  (Mathematical)  Model  Prediction  of  Performance  Decrement 

7.2  Script-Based  Inference  Prediction 

7.3  Information-Processing  Theory  Prediction  of  Processing  Overload/Underload 

8.  WHAT  ACTIONS  DOES  THE  ADAPTIVE  SYSTEM  INITIATE? 

8.1  Task  Allocation 

8.1 .1  Full-task  Automation/Manuat  Control 

8.1.2  Part-task  Automation/Manual  Control 

8.2  Task  Aiding 

8.2.1  Warning  Messages 

8.2.2  Problem  Restructuring 

8.2.3  Diagnostic  Help 

9.  HOW  IS  INVOCATION  OF  ADAPTATION  COMMUNICATED  TO  THE  PILOT? 

9.1  Explicit 

9.1.1  Message  Display 

9.1.2  Change  in  Pilot-System  Interface 

8.2  Implicit 

9.2.1  Transparent  to  Pilot 

10.  SHOULD  "CYCLES"  OF  ADAPTATION  BE  REGULATED? 

10.1  Short-Cycle  Regulation 

10.2  Long-Cycle  Regulation 


V  Preliminary  Design  Recommendation 
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